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ABSTRACT 

This report represents part of an investigation 
coordinated by a special steering Committee working under the 
auspices of the National council for the Social Studies. The purpose 
of Task 2 was to review independently the National 'sessment of 
Educational Progress Social Studies and Citizenship exercises as to 
the extent to which confidence could be placed in then. The major 
criterion for this confidence was the degree to which the test 
exercises have content validity or the degree to which the exercises 
are a direct measure of the objectives developed for the assessment, 
secondary criteria included whether the exercises could be utilised 
as models by teachers in developing their own evaluation instruments, 
cognitive and affective levels, age appropriateness, and 
appropriateness of the situation in which the exercises were 
administered. The results indicated that of the 19U exercises 
prepared for social studies, 85 percent (164) were considered valid. 
The majority of exercises were of proper age and difficulty level, 
although a few were too difficult for 9-year-olds and too easy for 
upper age levels. Of the 152 exercises prepared for citizenship, 93 
or 61 percent were considered valid. Only 55 percent of the exercises 
could be utilized as models by teachers while most valid exercises 
were of proper age and difficulty level. As a whole the citizenship 
exercises did not measure up as well as the social studies exercises. 
(DE) 
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VALIDITT OF SOCIAL STOIBS AND CXTIZSN3HIP S&5RCISE3 

mt IX 

Introduction 

Th« ttajor thxvst of this projdct coondasioned by the National Couneil 
for th« Sooial Studies was to analyzo the )ffational Assessxaont of Educational 
Progress aotiTlties idth regard to the developtsent^ preparation, and adain- 
istration of exercises in citizenship and social studies. Great attention 
has been on national assessment » and both educators and menibers of the lay 
public are interested in obtaining infomation that will enable them to 
make Judgisents regardiing the successes and contributions of National assess- 
ttinb* 

Primary Purpose 

The prime purpose of Task II iias to focus specifically on the exercises 
dereloped by the Educational Comndssion of the States in order to detendne 
the content validity of these exercises* Valid exsrcises are certainly 
necessary but not sufficient in procuring data as to the **state of affairs** 
regarding Americans* IstoIs of understanding and functioning in the areas of 
social studies and citisenship* 

Coiiteat validity is the most crucial criterion of any test exercise , 
for it appraises whether the exercise assesses what it is assumed to be 
measuring. Content validity centers on the representativeness or sampling ade- 
q^cy of the content of the items* Those concerned with detendng the content 
validity of test items or exeroiies generate the question <'Is the substance or 
content of the itsn deplotive of the content or the universe of content 
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boing Masured^CKerXlnger^ 1963, p. A46)« In the case of the KASP ex9r«- 
oises, the question is ^Art these exercises representatlre of the objectives 
as defslopsd by the Bduoatlon Comnlsslon of the States?^ Specifically, are 
the exercises doing %ihat they are purported to do in relation to the objee- 
tlyes? 

The detemining of content validity rests on the judgment of individuals* 
Specifically, the concept of content ^ralldity is based on the assumption 
that there is a theoretical universe of items that could be developed to 
test the defined content* It is impossible to randomly select items from 
this universe, for the universe is only a theoretical concept* Therefon», 
one most be satisfied with detemining the representativeness of the items • 
The question now becomes, ^Jb this exercise representative of the universe 
of items relating to & specific objective or area?" 

Since content validity relates to the question of whether some item is 
a direct measure of some Important bit of knowledge, skill or attitude in 
a subject area (Flnley and Berdie, 1970, p. 15), the task of determining 
whether exisrclses meet this criterion is difficult* Vfhen one is relying 
on the judgment of persons to determine whether exercises meet this cri- 
terion, one taways runs the risk that judges will disagree or agree but 
be off«-base. K.jw«ver, the incidence of non-agreement was reduced by se« 
leeting individuals possessing expertise in social studies education and 
eltiseashlp education and also having knowledge of questions, question forma- 
tion, and qiiestloning strategies. 

To facilitate the task the Bduoatlon Commission of the States provided 
the judges with detailed lists of objectives against which to consider the 
exercises • 
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Stoondary Purpose ^ 

In tddition to detendning the content validity of the exercises, the 
NAKP offices ifcre interested in deteradning if the exercises as developed 
could be tttiliied as models by teachers in developing their own evaluation 
instranents. Were these exercises suitable test exemplars in addition to 
being valid? Part of this concern involved the feasibility and workability 
of the exercises* It is possible for an exercise to possess content valid- 
ity, but be a poor exBiaple for cUssroom use, either because of its being 
too easy or too difficult, or not possessing sufficient challenge for indi- 
viduals, or not being feasible to adninister with regard to the context of 
the regular classrooou 

Ag^n, the appropilateness of the exercises for teacher use rested upon 
the judgments of the major investigator and the individuals on the panel 
which reacted to the exercises* 

Other Interests 

When one commences judging test exercises, one also begins to search 
for other aspects such as the cognitive and affective levels, the age ap- 
propriateness of the exercises, and the context validity or the appropriate- 
isess of the situation in which the exercises were administered* But, the 
reader needs to be cognizant that these concerns were subsidiary* Hope- 
fully, discussion of these additional concerns will provide readers of the 
report with a more total understanding of the exercises as used for assessing 
soelsl studies and citizenships knoidedge, attitudes, and skills* 
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Social Studiee. Cltlaenahip Goals and 
Obje^cilvee of National Aasesanent 



For determining the content validity, IWBP provided the judges \dth 
detailed listing of the objectives for both social studies and citizenship. 
To enable the reader to interpret more completely the report and to 
fonaolate independent judgments as to value of the overall thrust of assess- 
jBSnt in this area, the objectives for both of these areas are included in 
this report. The objectives have been draim from published documents of 
the Education Coasmission of the States and are stated verbatim. Detailed 
SttbHieetions and explanations of the objectives have been deleted to prevent 
unnecessary length of this report and duplication of docunsnfcs already in 
existence* 



Social Studies Obpleetivet* 



Social studies is that area of the school curriculius that 
seeks to conminicate about man in society. It is a shorthand 
tern for such subjects as history, geography, economics, po- 
litical sciences, anthropology, sociology, and social psychology. 
In some instances, social studies stands for an integration or 
coinbination of tw or more of these subjects, with or without 
an emphasis on conteniporary problems. But more often in grades 
5 throu^ 12, social studies tends to be simply a histoxy, 
geography, government, or economics course with materials 
adapted to the appropriate grade level. 



^Katioaal Assessment of Bdueational Progress (Education Commission of 
the States, Denver, 1976; pp* 9-^* 
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U Have Curiosity Aliout Hmn Affairs 



A* Baist quaations and seek answers. 

Are open to new information and ideas* 
C« Try to understasid why other people think and act as they do. 



IX* Use AaalytiC'^eientifie Procedures Effectively 



A* Identify and define problems and issues* 

B« Poraulate generalisations and hypotheses capable of being 
tested* 

C* Obtain inforwation from a variety of sources* 

0* Distinguish facta from opinions , relevant from irrelevant 
informationy reliable from unreliable sources*. 

X* Detect logical errors, unstated assumptions and uiMiarranted 
assertions; question unsupported generalizations; are aware 
of the complex nature of social causation, and understand 
that sequence or relationship does not necessarily iaply 
causation* 

fm Use data and evaluative criteria to make decisions* 



III* Are Sensitive to Creative-Izituitive Methods of S3q>laining 
tlie Human Coaditioa 



A* Read history, philosophy, and fiction* 

B* Obtain insight into human affairs from history and philosophy, 
and from fiction and other fonns of art* 

C* Recognize the role of creative-'intuitive methods in 
scientific inquixy. 

D« Distinguish personalized explanations of human affairs from 
8eientific<*objeetive explanations* 
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I\r* HaTe Knowledge Relevant to the ^fajor Ideas and Concerns of 
Social Scientists 



A. Understand soae of the distinctive modes of inquiry (ques- 
tions and approaches) of social scientists* 

B» Understand sens of the najor relationships involving culture^ 
the greup^ and the self, 

C* Understand some of the aajor characteristics of ecor^onde 
systens, etpeeialily the American econoodc system* 

D» Ifetderstand sone of the m^or characteristics of the geog- 
raphic (spatial) distribution of nan and his activities » 
and of nn's interaction with the physical environaent* 

£• Understand som of the najor historical developnents* 

P« Understand soae of the characteristics of the najor systens 
of govemnent^ paztieularly the political system of the 
United States. 



V« Have a Reasoned Commitment to the Values that Sustain a 
Free Society 



A* Believe in the fundamental worth of the individual and can 
justify their belief* 

B« Believe in the freedoms of the First Amendment and can 
justify their belief. 

C» Believe in the rule of law and can justify their belief* 

0. Believo in open opportunity for advancement and can 
justify their belief* 

B* Are willing to act for the general interest* 

F* Are willing to participate in decision making relevant to 
their lives. 



Under each objective are detailed descriptions of knowledge and/or 
behaviors that should be possessed or evidenced by individuals at various 



«g0 ImlB. Some of these objeetives had lengthy discussions of sub-under- 
standings or sub-behaviors 0 In determining the content validity for each 
item> the Judges scrutinised these descriptions under the najor objectives. 

Citisenship Objectives 

The Bducation Conndssion of the States did not create a precise defi- 
rJLtion of citizenships however^ their interpretations of citizenship are 
evident through the stated objectives. It should be indicated that some 
areas coanonly considered to be related to citizenship^ e.g. knowledge of 
nays of life in other cultures^ iiere not included under citizenship* but 
rather were incoxporated in the social studies area. Often in social 
studies education* citizenship and social studies objectives are melded* 
vbut the Coanission did divide them. 

Another point for the reader^s consideration is that the NABP personnel 
instructed the investigator to consider both the citizenship objectives 
and also the major citizenship goals as de^loped for an early report 
(MABP, 1970). 

Therefore* both th» goals and the objectives are included at this 
Juncture of the report. It is evident that the goals and objectives are 
extremely similar* 

Citizenship Ob.leetives* 

^tional Assessment ofBducational Pro^yess. Report 2. Oitizenship t 

M ational Results (Bducation Commissien of the states. Denver ^ « 
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TASK II 

1. Show Concern for the Wtoifare and Dignity of Othex's, 

A* Treat all individuals nith respect 

B» Contlder the eoasequences for others of their own actions, 

C. Quard safety and health of others* 

D, Help other inditiduale Toluntarily. 

S« Are loyal to country, to friends, and to other groups vhose 
ral\m they share. 

P. Understand and oppose unequal opportunity in the areas of 
education, housing, eaployment, and recreation. 

G. Seek to iwprove the welfare of groups of people less fortunate 
than they* 

II, Support Rights and Freedoms of All Individuals* 

A« Understand the value of constitutional rights and freedoms* 

B« Recognize instances of the proper exercise or denia? of 
constitutional rights and liberties, including due process 
of lair* 

C. Defend rights and liberties of all kinds of people uniformly, 

III* Help Maintain Law and Order 

A* Understand ths need for law and order* 

B« Are conscious of right and wrong behavior. 

C* Cowply with public law and school rules, 

D, Help authorities in specific cases* 
R« Protest unjust rule^ openly 

y. Infora theaselves aboi^t the law* 
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Vi, Xnoir the Main Structure and Functions of our CJovernatents 



A* Riseognise the purposes of governraont, 

B, Recognixe the Mtin functions and relations of go^jemiyntal 

C, Recognize the importance of political opposition and diverse 
interest growpe, ^ 

D, Recognise that denocracy- c ;..pends on tb« alertness and involve- 
»ent of its citizens, and ':<:now how citizens can affect govern- 
aent. / 



K, Beeognise the structure and ?>peiation of political parties. 
F» Know structure of se^Sbol and/Mudent gover2a?«nb , 



fi' 



V. Seek CoBmunit»fIaprw^;i»nt Through Active, D^awcratic Parti- 
cipation. 



A. Believr^hat w«Sh person's civic beh&vior is important, and 
cenvi^this ifSlief to others. 



B, »pgnizyi«portant civic problena and favor trying to solve 
/phem# ^ 

/ / 

M Actively work for community improvement. 

0. .Participate in local, state and national governmental processes 



• / ^ ^PP-''^'' <i«"ocratio procedures on a practical level Khen working 

/ / in a group. 

^ F, Display fairness and good sportsmanship toward others. 

7I» Understand Problems of International Relations 

A* Are aware of the problems of international conflict and dangers 
to national security. 

B» Seek world peace and freedom for all peoples. 
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VII. Support Rationality in Coanaunication, Thought and Action 
on Social ProblOBHi* 



A« Trj to inform thmselves on socially important matters and 
to understand alternative viewpoints. 

B. Evaluate coimunioatlons critically and form their oim 
opinions independently* 

C. Weigh alternatives and consequences carefully, then make 
decisions and cany then out without undue delay, 

D. See relations aaong social problems and have good ideas 
for solutions. 

S. Support free communication and conwunieate honestly with 
others, 

P. Understand the role of education in developing good citizens. 



VIII. Take Responsibility for Own Personal DevelepMnt and 
Obligations. 



A» Further their own self-improvement and education. 

B. Plan ahead for major life changes. 

C. Are conscientious, dependable, self -disciplined, and value 
excellence and initiative. 

B. Eeonomieally support self and dependents. 



IX, Help and Respect Their Own Families (Ages 9, 13, 17) 

A. Respect the reasonable authority of their parents or guardians, 
and help with home duties and problems. 

B. Help younger brothers and sisters to develop good citisens. 

C. Discuss social matters with their faioilies and respect the 
ri«m of all family msifbers. 



TASK XI 



IX. Nui»ture the Dev»lopjiBnt of Their Children as Future Citizens 
(Adults) 



A* Provide for the basic needs and health of their children. 

B. aicourage cooperative, ethical relations to authority and 
to other individuals. 

C, Develop in their children & broadening awareness, independence 
and rationality. 



Cltigenahip Gcals * 

A. Show concern for the well-<being of others. 

B. Support rights and freedoae of all individuals. 

C. Recognize the value ef just law. 

D. Know the »ain structure and functions of government. 

E. Participate in effective civic action. 

F. Understand problews oi^ international relations. 

G. Approach civic decisions rationally. 

H. Take responsibility for own development. 

I. Help and res'peet their own fasdlies. 

Each of these goals was described in sette detail which provided the 
judges with additional information for deteradning the exercises* content 
validity. 

^national Assessment of Eaueatienal Progress » Report 2 . pp. vil « ix 
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Th« 8p»cif le Mature of the Task 

Gl«ner«l Badcground Discussion 

As prsTiously indicated, the Task n investigator was to review inde» 
pendenbly the NABF social studies and citizenship exercises in order to in- 
form KASP| school personnel, and the general public as to the extent to 
which thejr can place eonfidsnce in the exercises. For purposes of orients* 
tion, it may be productive to discuss briefly the exercises in both social 
studies and citixenship. 

First, the term **exereise8i* is tised to distinguish these test item from 
test itens iq)pearing in standardised tests. The iiriting of these exer- 
cises was sub-contracted to private measurement agencies, who developed 
exsreises relating to the objeetives in ten subject areas. These exercises 
were to receive two reviews t by liy people who would ascertain that no 
exerdse nas potentially offensive to any large group of people and by ex- 
perts in each subject field who responded via mail as to whether they 
thought each exercise did asssss the educational objective for which it was 
writtsn. After these ti^ reviene were completed and indicated revisions 
carried out, exsreises with different formats were tested out for ease of 
administration. 

It is not the purpose of Task II to provide a history of the develop- 
ment of these exercises, and those individuals desiring more depth of treat- 
ment regarding this aspect can read other section of the overall NG33 
report or refer to MAKP^s booklet, The Natipnal Assessment Approach to 
Bxji|ii|<jise. Pevelopmettb ^. 
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Pormata of Bbwrelses 

Originally nine exorcise fomats xere considered (Pinlegr and Berdle^ 
1970). However, NABP decided in the first assessinent to use on3y four 
foraatat 1) completion^ 2) writing^ 3) nviltiplenshoice^ and 4) cooibina- 
tion. Kx»nplee of each of these are preeen'ted below. 



ConpletioA (Finl^jr and Berdie^ 1970« p. 20) s 

This type of exercise requires the student to fill in 
1])laidc(a) or to provide a short answer following some stimulus 
or to arrange alternatives in correct locations. 

Wukt are the capitol cities of the foIlowLng states? 

Ksw Raapshire 

Oregon 

TexM 

Georgia*" ~^ 



HMting (Finley and Berdie^ 1970« pp. 20-21) t 

The response to this type requires the student to write a 
senteneeCs) or paragraph(s) or list responses. 

ICrlte a set of directions explaining how to get, by car or 
on foot, to eone fanous local landmark from the airport, 
railroad station, bus station or turnpike (freeway) exit 
closest to where you live. 

WMte your directions carefully and clearly, as if you were 
going to give thea to a friend ilio is not very fasniliar 
with your area. 



l&atiple-choice (Plnley and Berdie, 1970, p. 23) t 

The correct response Is selected from two or more 
alternatives. 
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Oireetionst Mix^ onlj one ansiier. 

John eo\uited his breathing rate several tines during two 
days. His record is shown below. 

Breathing Rates 

Tine of Dsj 

P«>y , yibndm , Weoi^ Night 

Tiesdigr 19 16 14 

Wednesday 20 16 13 

Wh^n HAS John^s breathing rate highest? 

.. Tuesday in the morning 

Tuesday at noon 
. , . Tuesday at night 
^^^^^ Wednesday in the morning 
_^ Wednesday at night 



CeMbination (Fii)l«y and Berdle^ 1970, p. 24) i 

The eoMbinatioi £om is a nultiple-ehoiee exercise folloned 
either by coa^letion or writing* 

Directions: Vtak your answer to the first question. If 
you narked «y»»" ansi^ the second question briefly. 

Hate you ever bean in the city where your state capitol 
is located? 

, Tes 
, No 

(If yes) What one thing that you saw in that city do you 
renefiftMr mt? 



Ages Considered for Bxere^ses 

ficercises were designed for f«ur agesj 9, 13, 17 and Adult. Howerer, 
■any of the exercises reviewed were appropriate for more than one age group* 
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Those exereiies generally overlapped bett<een two adjacent age groups, but 
there were exerelsee where the overlap mm over three and even four age 
groupe* In eone instaneeai some exereieee overlapped b0ti«een two non-adja- 
cent age gro\q^, e.g. 13 year olde and adults* 

AdndnAetratlon of the RaBrdaes 

Another aspect of the exercises taken into account was the manner of 
adninistration, Sbcercises were given either to individuals or to groups* 
For the individually administered exercises, examiners interviewed students, 
ifoxking idth a small nierophone pla^sed inconspicuously between the exa2idner 
and the child. The intertiewers infomed individuals that what they had to 
say was inqportant evidence* The tape provided information so that the ad- 
ministrator did not have to write all the conversation. 

Qroup administered exercises were assembled in packages to be taken by 
individuals. In most eases » the administration procedures as well as the 
•xerclses in the actual assessment packages were on tape* 

Procedure for Processing the Exercises 

The Investigator and three panel members processed the exercises to 
estimate whether they measured the objectives intended and to record descrip- 
tor information on each exercise dealing with age group, format, manner 
administered and the like. 

The investigator's initial step was to develop a format sheet for 
recording pertiiiient data. This sheet, lAiich is found below, allowed for the 
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Figure 1 



FORM/IT FOR RBCJORDING BaCERClSE DATA 



8iA>Jefit areat 
ItaM! 

Individual or group adalnistereat 
Ag«: 

ObJeebiT»s 

Cognitivd/affeetiTtt lavelst 
Content validity: 

Contoxb valldityt 

Model for teaehert 

Ace appropriateneees 

Other: 
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recording of subject area, the specified exercise, the exercise ts format, 
how administered, the age of persons dealing with the exercise, the objec- 
tlye to Khieh this exercise was geared, and judgment regarding content 
-validity. The data sheet also enabled the recording of the cognitive and 
affeotlTe levels, the context validity of the exercise, the exercise's 
model qualities for teach(»r use, and finally the exercise* s appropriate- 
ness for the intended ages. As can be discerned, part of the infom»tion on 
the sheet simply described a particular exercise, while the remainder of the 
sheet dealt idth the dlMnsloss of the exercise judged by the investigator 
and three panel meobers. Bach member used this format in reacting inde- 
pendently to exercises. 

The sequenticd Instructions folloiied for processing both the social 
studies and citizenship exerclsas were identical: 

1* Read each exercise and record descriptor ihfon&atlon, 

2* Mttke judgments as to iifaether the exercise had content validity, 

context validity, exemplary qualities for teacher use, and age 

appropriateness , 
3* Read appended responses to each exdrcise if present, 
4* As a safeguard against ods judgment reconsider the content 

validity question id.th regard to how individuals actually 

responded. 



These steps were adhered to by the major investigator and also by 
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throe judges* invited to participate In detemdning the exercises' con- 
tent validity. After responding independently to aXl exercises » the judges 
met as a group idth the major investigator to discuss their valuations 
regarding th? content validity and the exercise's exemplarity qualities. 
Other information relating to context validity, cognitive/affective lev- 
els, and age appropriateness also were considered» 

The investigators and the three judges discussed at length any 
exercise where there was sobs disagreement as to content validity. 
Reasons for the dlffSi-lng interpretations were contemplated. After such 
discourse, the content validity was reconsidered for the exercise in ques- 
tion. The investigator put aside for reconsideration and more detailed 
analysis those exftroises where consenstss could not be achieved. Later, 
the major Investigator made a second judgment regarding these exercises' 
content validity. Where there was some disagreement regarding some other 
aspect of the exercise such as the affective level, the major investigator 
did not deem a careful reconsideration necessaxyj in such instances he 
used his own interpretation. This tack was taken because these other 
concems were not the major ooncextis of Task IX. 

*ife. Thresa fi. Gieger, Instructional Specialist, Portland Public Schools, 
Portland, Oregon; Arthur Z. Ware, Social Studies Coordinator, Bellevue 
Public Schools, Bellevue, Wisshington, Mr. Jack S. Thonqwon, Social Studies 
CoordiiKttor/Speciallst, Benton School District, Renton, Washington. 
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After the sharing of perceptions by the major investigator and the 
other judges, infomation was then organized in tables to indicate the 
basic infornation. Tables then were organised according to the major eb- 
jectite concerned. For instance, there would be a table for recording all 
pertinent data relating to objective U, while all data relating to oxer- 
eisee concerned with Objective IIB would be recorded on another table. 

Kxercisee were divided by NAEP into both social studies, released, and 
unreleased and citizenship, released and unreleaaed. To prevent ai^ aix-up, 
the exercises were prooeseed in these divisions; for exaaple, all judges 
processed social studies exercises released as a group, then judges considered 
all social studies objectives unreleased. The procedure was repeated for 
the citizenship objectives. Judges gave careful attention to indicating 
on the data sheets the area and status of each exercise. 

Some of the released exercises in both social studies and citi25en8hip 
are present in appendices. However, unreleased items cannot be identified 
At this ti«e as these exercises will be administered at sone time in the 
future. 

The Jydftfip 

The three persons who served as co-judges on the content validity of 
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the exinises were well qualified to make such judgments. All three indi- 
Yiduals hold positions of leadership in the social studies curriculum area. 
Two individuals are currently district social studies coordinators and have 
had a great deal of experience in curriculum development and program as- 
sessment. One judge this past year has been on leave from her staff posi- 
tion to engage in doctoral study* 

Resumes of each of these individuals act found in the appendix of thie 
report. Suffice to say that these people are knowledgeable of the area of 
social studies, of curriculum in general, and have expertise in woricing 
with questions* 

Perhaps at this juncture some information is necessary on the major 
investigator for Task II. Detail is provided in a resume in the appendix. 
However, this investigator has worked in the areas of social studies and 
general curriculum for the last seven years. For the last ten years, he 
has been concerned with the area of questions, the effects of questions en 
ehildren*8 thinking and achievement in social studies. He has condueted 
research on questions and has written numerous articles dealing with ques- 
tions. He has authored a book dealing with questions and has consulted 
with school districts across the nation conducting workshops on questions. 
He also has developed questions for Educational Testing Service. 

One might find this reporting of credentials somewhat unusual in a report, 
but since content validity is couched essentially in the judgment of an in- 
dividual, one does need some data on that individual to determine if he has 
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the necessary expertise and depth of understanding to make such judgmenls aa 
to th# represfinMivensto or sampling adequacy of the esicepciscs in social 
studies and eitisenship* 

The Results 

Results are reported in four divisions: results for released social 
studies exercises, for untreleased social studies exercises » for released 
eitisenship exercises » and for unreleased citizenship exercises. The reader 
should be rAiindful that the central thrust of the Task II investigation was 
to deteradne the content validity and the exemplary qualities of these 
exercises^ but other diaensions of the exercises were considered and are 
included in the followinf taibles. 

The data are reported for each objective. General sunsftries of all 
the social studies and citisenship exercises, released and unreleased^ are 
present in the discussion section. 

Social Studies Exercises, Released 
Objective I* Have Curiosity About Huaan Affairs: 

The first sub-Klivision of objective one, I, A is specifically concerned 
with whether individuals raise questions and seek answers* Table 1 for 
social studies released reports the data relating to this objective. As 
can be ebeerved there were six exercises designed to test this objective. 



Tabl e 1 (Exercise Are a SSfi ) 



Total 
Items 




Item Fortoat 


Adtuitilstered 


Age Levels 


Coe. Level 






Wri. 


Ml.Ch. 


Comb. 


Ind. Group. 


9 


13 


17 




s 




(> 




1 




5 




3 


3 


2 








1 


1 






Aff. ] 


Level # Valid 


Hot Vali^. 


Exetn* 


// Con.. Val.. 




mm 


m 














3 


2 






5 


1 






4 



Panel nenftjers judged fiTS exercises to have content validity and one as 
not v^alid* IIovieTer, meidbers considered all exercises to be good examples 
for teaoher x»e and to have context validity and appropriateness for the 
ages inaieated. One nay iionder how an exercise considered not valid can 
have context valiiity or be extnplary. In many cases, exercises were 
judged to lack content validity with regard to the objective indicated, 
but Here still eoi^idered good escereises for another objective. Therefore, 
a teacher could utilise the exercise or a similar one if she selected that 
objective. Also, the context validity referred to the type of situation 
in which the exercise was administered as well as the type of situation 
implied by the exercise to which the individual had to relate. Oftentimes, 
items were not valid regarding the objective but were valid regarding the 
situation implied or specified, given another objective* 

*The cognitive levels used were drawn from Benjamin Bloom, ed. , Taxonony 
of Educational Ob^leetives . Handbook It Cognitive Domain, New Yorkt David 
MeKay Co», 1956, while the affective levels were drawn from Darld Krathwohl, 
•d>, Tawnoffl of Educational Obieetivis. Handbook lit Affective Domain, 
Msw Tortct David MsKay Co., 1964. 
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The cognitive and affective levels \yere attended to when panel members 
and the investigator could make a judgment. Oftentimes, items had no 
3lear37 discemable affective component* 

t^pon analysis of the table, one can note that the nunibera for age level 
and cognitive l*vel do not always equal, and sometimes exceed the total 
nuabep of items. This is due to the fact that many exercises were appropri-. 
ate for more than one age and that oftentimes, no judgment vbb deemed 
possible regarding the cognitive and affective levels. 

The second eUb-division of objective one, I, B, relates to whether 
individuals are open to new information and Ideas. Table 2 (Social Studies 
Released) presents these daU. 

Tabl e 2 (Exercise Area SSR) 



Objective: 

IB 



Total » 
Items 



Comp. 



Item Format 



Wrlii'Ml.Ch. 



tomb. 



/ 



Administered 



Ind> Group 



A ge Levels 



13 



1 



17 



Ad. 



K 



Cog. Level 



An 



1 



Aff. 1 


jevel 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


^ '■■ " ' 

Age App. 
























1 






1 






1 



Oaly one exsrolse asasur<Ni this objective and this item was judged not valid. 

1>C the third sub^vlsioa of objective I relates to whether individuals 
nry to understand why other people think and act as they do. « Table 3 
(8SB) reports these data« 
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Table 3 , (Exercise Area S5R ) 



Objectiv 

Ic 


e: 


Total 
items 






Item 


Format 


Administered 


_ A{(e Levels 


Coe, Level 






Wri. 


^a.Ch. 


Conbt 


Ind. Group. 


9 


13 


17 


Ad 


K 


C 


A 


An 


S 




2 




1 




1 


2 




1 


I 


1 






2 










A£l 




Level 


// Valid 


J(ot Va 


lid. 


# Exem. 


Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 














1 




1 




2 




2 


} 


2 





As can be discerned, two items, both yalia, existed for this sub-objectiv*. 
The investigator and panel also deemed these exercises to be exen^lary and 
to have context validity and to be appropriate for the age levels indicated. 

Objective II. Use Analytic-Scientific Pi j-jedures Bffectivelyt 

This objective had sdjc stib-divisions, the first dealing with whether 
individuals "identify and define problems and issues** (II A). Data relating 
to this sub-objective are indicated in Table 4 (SSR), 

Table 4 (Exercise Area SSR ) 



ERIC 



Objective: 

11 A 



Total j!f 
Items 



tionp. 



Item Format 



WriUHl.Ch. 



1 



3 



Comb. 



Administered 



Ind. Group 



Age Levels 



13 



17 



Ad. 



K 



Cog. Level 



1 



A An 



SIB 



Aff. Level 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


Age App, 


Re 














1 










3 


1 


2 


2 


3 
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Review of the table reveale four exercises designed to measure this objec- 
tive with three being judged valid. However, the investigator and panel 
judged only two to be of exemplary quality. 

"Formulate generaHaations and hypotheses capable of being tested" is 
the seoond sub-division, II B. There were no exercises released to aeasure 
this objective. The next sub-objective measured was II C, relating to 
whether individuals "obtain information from a variety of sources. ♦« Table 
5 presents these data. 

Tabl e S (Exercise Are a SSR) 



Objective: 

n c 



Total 9 
items 



Comp, 



Item Format 



Wri, 



MjLCh. 



Coab. 



Administered 



Ind« Group . 



Age Levela 



11 



17 



Ad 



Leve; ■ 



13 



7 



1 



Re 



A££. Level 



i_yalld 



1 



a No t 



1 



11 



// Con. Val. 



12 



There were thirteen iteas measuring this objective, twelve of which the 
judges determined to possess content validity. However, only eleven items 
<!ttalified as exemplary. It is interesting to note that in these exercises, 
aljiost all levels of the cognitive domain were represented. 

Table d (SSR) deals with informtion relating to the fourth sub-division 
of objective II. Speeifieally this objective, II D, relates to measuring 
nheth^p individuals -distinguish facts from opinions, relevant from irrelevant 
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Objectiv 

ilD 


e: 


Total 
Items 




Item Format 


Admin :^.8tered 


Age j 


Levels 


Coe. level 




comp. 


Wri:J 




Comb. 


Ind. ' Group 


9 


13 


17 


Ad. 


K C 


A An 


S 


E 


5 






5 






5 


5 


3 


2 


2 


1 


2 






I 




Afl 




Lev< 


si 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


Age App. 




Re 


R8 


Va 


Or 


Ch 












1 










3 


2 


4 


4 


5 



4«iiigiwd for this objective} however only three wre judged valid and four 
exenplaty. 

The next aub-objective of objective II deals with whether individuals 
"detect logical errors ^ unstated assumptions, and unwarrantod assertions; 
question unsupported generalizations; are aware of the complex nature of 
social causation and understand that sequence or relationship does not 
necessarily duply causation. Table 7 reports these data for objective II E, 



Tabl e "7 (Exercise Area SSR ) 



ERIC 



Objective: 
II E 



Total fl\ 

items Icomp.l Wri. 



Item Format 



Ml»Ch. 



Coab. 



Administered 



Ind. Group. 



Age Levels 



1 



13 



n 



M 



1 



Level 



ALAn 



Re Rs 



Aff . Lev el 

1 



// Valid 



\\ Con« Val. 



AgeA£g. 
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map administered five exercises for this objective and the investigator 
and panel detewdned four to have content validity. Hoi^ever, all exercises 
nere judged exeapXary. 

Ko exercises measured the last sub-objective of this division, Objec- 
tive II "Use data and evaluative criteria to make decisions •» 



Objective III, Are Sensitive to Creative-Intuitive Methods of Explaining 

the Human Condition! 

The first division (HI A) of this objective aims at measuring whether 
individuals tread histoiy^ philosophy, and fiction." Table d presents these 
data* 

Tabl e S (Exercise Area SSft) 



Objective: 

niA 



Total 
Items 



Item Format 



Administered 



1 



1 



Ind» Group 



1 



Aee Levels 



13 



1 



17 



1 



Ad. 



Cog. Level 



K 
1 



A 



An 



Aff 



Re Ks 



Va 



level 



Or Ch 



SIE 



// Valid 



// Not Valid 



// Exem. 



// Con. Val. 



1 



1 



Only one exercise existed for this sub-objeetive>and the investigator and 
panel assistants judged it valid and exemplary* However, the judges con- 
sidered the item inappropriate for all the ages intended; specifically those 
behaviors specified for ages 17 and adult ^re not measured by the exercise* 
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Table 9 (SSR) has the data relating to objective IltB, ^obtain insight 
into human affairs from history and philosophy, and from fiction and other 
fonw of art. » There were four exercises for this objective and all 



Tabl e 9 (Exercise Are a SSR ) 



Objective: 



TotaO 
items 



4 



Item Format 



Wri. 



Ml.Ch. 



.Comb* 



Administered 



Ind. Group. 



Age Levels { Cog . Level 



13 



17 



4 



Ad 



An 
4- 



A££. I 


.evel 


// Valid 


// Not; VftUA- 


^ Exem. 


/; Con. Val.. 


Age Apj.. 


Re 














r .1, , 


1 




2 


1 


4 




4 


4 


4 



qualiflMl as valid and exsmplaiy. Note that these exercises involved some 
of the affective domain levels* 

Suib>6bjeetive IIIC» *'reoognisse the role of creative-intuitive methods 
in scientific inquiry was without exercises* The nexfe exereiies were 
ooncemed with Sttb-K>bjective IHD, whether individuals "distinguish person** 
alixed e3Q>lanatio}ks of htttian affairs from scientific objective explanations.* 
Data relating to this objeetive are recorded in Table 10 (SSR)« 



Analysis of the table reveals a total of three items, all possessing 
content validity and being valuable as models for teacher use* 
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Table to (Exercise Area SSR ) 



Objective J 

DID 



Total // 
Items 



Cony. 



1 



Item Format 
WriU-Ml.Ch. 



Comb. 



Administered 



Ind» Group 



Age Levels 



13 



17 



Ad. 



1 



Cog. Level 



K C 



A An 



1 









f 




Afi 




Level 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


Age App.^ 


Re 


Re 


Va 


Or 


Ch 
















1 






3 




3 


3 


3 



Ob>«tt«« IV, Have Knowledge BeXevant to the Major Ideas and Concerns of 
Social Sdf Kbists 



The first sub-«>bje«tive of this division relates to measuring whether 
individuals ^understand some of the distinctive modes of inquiry (questions 
and approaches) of social scientists •** Table 11 (SSR) reports these data. 



Tabl e 11 (Exercise Area SSR ) 



ERIC 



Objective: 

lYA 



Total # 
itens 



1 



COMP. 



Item Format 



Wri. 



Ml«Ch« 



1 



Coiiib • 



Administered 



Indr Group. 



1 



Arc Levels 



9 13 17 Ad 



1 



1 



K 



Cog.. Level 



1 



An 



, Af£. Level 


// Valid 


A Not Valid. 


^ Exem, 


l> Con. Val,. 


Age App. 


mm 


m 


cam 






















1 




1 


1 


1 
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As eaa be discerned, only one exercise existed to measure this particular 
objective. The investigator and panel judges deemed it valid and exemplary, 
as well as appropriate for the ages indicated. 

Objective VT B measured nbether individuals "understand some of the 
major relationships involving culture, the groi^, and the self," Table 12 
(SSR) has these data. Observation reveals that three exercises were pre- 
pared for this objective and that the investigator and panel mesibers 
indicated tiio as valid. 



Tabl e 12 (Exercise Are a SSR ) 



Objectlv 

IV B 


e: 


Total 
Items 


if 


Item Format 


Administered 


Age ] 


Levels 


Cos. Level 




Conp. 


Wri;j 


/Ml* Che 


Comb. 


Inde Group 


9 


13 


17 


Ad. 


K C 


A 


An 


S 


IE 


3 




1 


2 






3 


1 


1 


1 


1 


J 


1 


] 










A££. 


4ev( 


iX 


9 Valid 


// Not Valid 


// Exem. 


// Cos.. Val. 


Age App. 


Re 


Rs! 


Va 


Or 


Ch 






















z 


] 


2 


1 


1 



T«ble 13 (SSR) reports the data relating to the next sub-division of 
this objectiv», 17 0, •♦ujideritand aoue of the major characteristics of 
•eonoaio STBtmw, espeeiaUgr the Aasrican economic sTstem.^ Sight exisrcises 
•ad/ited for this objective, six of trfiich wre valid and exemplary. 
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Tabl e 13 (Exercise Area SSR) 



Objective: 

lYc 


Total It 
items 


Item Format 


Administered 


Arc Levels 


Cos. Leve] 




Comf>i 


Wri. 


Ml.Ch. 


Conbt 


Ind. GrouD. 


9 


13 


17 


Ad 


K 


c 


A 


An 


S 














9 


3 


3 


5 


5 


z 


4 


z 






Afl 


!. 


Level 


// Valid 




# Exem. 


Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 












1 










6> 


z 


6 


7 


7 



Table 14 reports tht data relating to the next sub-obj«etiv« under 
objectiye IV, "Understand sene of the major characteristics of the «eo- 
.STftphie (spatial) distribution of man an4 his aetivities^ and of Man's 

Table 14 (Exercise Area SSR) 



Objective: 

IVd 



Total 
Items 



tomp* 



Item Format 



WiT 



>Ml«Ch* 



Comb. 



Administered 



Ind. Group 



Age Levels 



1.3 



17 



Ad. 



Cog. Level 



K C A 



} 1 



An 



E 



Aff. 1 


»tevel 


// Valid 


II Not Valid 


// Exem. 


# Con. Val. 


Age App. 














2 






1 


1 


7 


2 


7 


7 


7 



interaction with the physical •nvironratate** Nine exercises nere constructed 
8«T«n of uhich mnt -ralid. One can note from the table that exercises 
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osdsted at four levels of the cognitive domain as weU as three levels of 
the affective domain. 

Objective 17 S related to measuring if individuals "understand some of 
the major historical developj»nts." Table 15 (SSR) reports these data. 
There yttn numerous exereisea (1$) relating to this sub-objective. Of the 



Tabl e 16 (Exercise Area SSR) 



Objectiv 

We 


e: 


Total if 
items 




Item Format 




Administered 


. Aee Levels 


Cos 


L. Level 


Comp. 


Wri. 


^a.ch. 




Ind« GrouD. 


9 


13 


.17 


Ad 


K 


c 


A 


An 


S 




15 




2 


13 




Z IZ 


7 


/o 


31 


11 


9 






] 






A£i 




Lev« 


il 


it Valid 


Not Valid. 


^ Exem* 


// Con. Val.. 


Age App. 


Re 


R8 


Va 


Or 


Ch 












1 


J 




1 




il 


3 


12 


12 


/o 



total, twtlve were valid and exemplary. However, the investigator considered 
only ten of the items appropriate for the ages intended. The major eoneem 
was that the fire items in question were p^rha^t too difficult for the 
nine-year-olds. 

The last aub-objective (IV P) measures whether individuals "undewtand 
some of the characteristics of the major systems of government, particularly 
the political system of the United States. Data relating to this objective 
•xlst in Table 16 (SSR), Eleven items were developed, and nine ranked as 
valid. It is interoating to note that the investigator and panel classified 
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Table \io (Exercise Are a SSi^ ) 



Objective: 

IV F 



Total id 
Items 



n 



Comp. 



Item Format 



Wrt;iMl.Ch. 



10 



Comb. 



Administered 



Indt Group 



1 



10 



Age Levels 



13 



17 Ad 



Co^. Level^ 



K.CtAM S 



5h 



Re 



Aff. Level 



Rs 



Va 



Or 



Ch 



// Valid 



// Not Valid 



// Exem. 



# Con. Val. 



Age App. 



•nly s«v«& exercises as having context validity; that is, appropriate situations 
in which individuals could consider the exercises or situations iaiplied or stated. 

Objective ?. Have a Reasoned Coanitnent to the Values that Sustain a Free 
Society 

This objective bad six sub-divisions, the first of nhich was "believe 
in the fundanental worth of the individual and can Justify their beUef.« 
Table 1? (SSR) contains data relating to this objective. Six exercises 

Tabl e 11 (Exercise Are a SS|<) 



Objective: 

VA 



Total # 
items 



Comp* 



Item Format 



Wrl. 



Ml.Ch. Comb. 



Administered 



Ind« Group 



Age Levels 



13 17 Ad 



Cok.* Level 



An S 





Aff, Level 1 # Valid 






# Con. Val.. 


Age App. 


Re 


















3 




2 






6 
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eadated for thla obJeotiTe imd all wre valid. These ejcercises also vtere 
rtnksd as being eawiplaxy and possessing context validity and age appro- 
pxlatenMS. 

"Believe in the freedoms of the First Amendment and can Justify their 
belief" m the second stAnobjective under V. Table 18 (3SR) reports 
relevant information. Eight exercises, all. deemed valid, existed. Hov^ever 
the inffeetigator and panel nesgbers considered onJy seven exea?)lary. 



TableJX. (Exercise Area SSR ) 



Objectit) 

VB 


'Bi ' 


Total # 
items 




Item 


Format 


Administered 


Age Levels 


Coe* Level 




Wri. 


Ml.Ch. 




Ind. 1 




9 


13 


17 


Ad 


V 
a% 






An 


s 


4 


9 




1 


Z 


5 


3 


5 


] 


4 




7 




PI 




1 


3 





Aff, 1 




# Valid 


// Not Vall4. 




# Con. Val.. 
























7 


y 




7 


1 


7 



Obj«etiv« 7 C aeaauTMi iihether individuals "believe in the rule of lav 
and can justify their belief." As can be observed from Table 19 (SSR), 
only one exercise tms developed and it vt,r^ considered vaUd. 
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Ks> amrelsM utre pMsenb for the fourth »ub-^vi8ion of Objectite V, 
"Boliof* in opon opportunity for advancement and can justify their belief •« 
However, there were thre« exercises for Objective V B, «are idlling to 
act for the general interest •« Data relating to this sub-objective are 
reeorded in Table 20 (SSH), Scanning the table reveals five vaUd exercises 



Tabl e 2>o (Exercise Area SSR) 
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Objectiv 

Y E 




Total 0 
items 




Item 
Wrl. 


Format 
^a.Ch. 


Coatb. 


Administered 






Leve 


Is J 




^el 




5 




Z 




3 




1 


9 
1 


,1,3 

2 


17 
2 


2 


1 


1*1 

1 


1 


1 


3 






Aff, ] 


Lev( 


ll 


# Valid 


Not Valid. 


. ^ T!xem. 


// Con.^ Val.. 




Re 




lOr 


Ch 
















2 


3 
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5 


5 
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for this objective. It should be noted that the investi/;ator and panel 
members placed three of the exercises at the synthesis cognitive level 
and at the upper categories of the affective domain. 

The last sub-objective is "are willing to participate in decision 
waking relevant to their lives." Table 21 (SSR) reports information 
relevant to these exercises. 



Table 21 (Exercise Are aSSR ) 



Objective: 


Total 
Items 


// 


Item Format 


Administered 


Age ] 


Leve 


Is 


Cog 


Level 




Comp . 


wri;j 




Comb. 


Ind. 


Group 


9 


13 


17 


Ad. 


K 


C 


|A 


An 


s 


E 


I 








2 




2 


] 


\ 


1 


1 








! 




1 


Afl 


c. 


4evel 


// Valid 


// Not Valid 


// Exam, 


ft Con. Val. 


Age App, 


Re 


Rs 


Va 


Or 


Ch 
















J 






2 




2 


2 


z 



Consideration of this table indicates two items for the objective and each 
judged as possessing content volidity and being exemplary for teachers. 



Social Studies Exerciser, Unreleased 

Since the unreleased social studies exercises are related to the same 
objectives as the released, there is no need to repeat in detail the objec- 
tives. Rather the objectivee w'll be referred to by their outline organization, 
e.g. I, Aj I, Bj n, A; 11 , V, P. 
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Objective I. Have Curiosity about Human Affairs 

Objective I A data are recorded it Table 22 (SSU). Note that tables 
are identified by SSU coding to distinguish them from the tables presenting 
social studies released data. Survey of Table ?.2 reveals four items present 
for this objective and that judges categorized all as valid. 

Table ZZ (Exercise Area SSU) 



Objective: 

lA 



Total il 




Item 


Format 




Administered 


Arc 


Levc 


.B 


Cor. 


Level 


items 


Comp^* 


Wri. 


^a•ch• 


Comb. 


Ind. < 


Group. 


9 


13 , 


17 


Ad 


K 


C 


A 


An 


S 




4 




3 




3 


4 




3 


z 




1 


J 


3 











A££. Level 



Re 



Rs 



1 



Va 



Or 



Ch 



// Valid 



U Hot Valid 



# Exem. 



// Con. Val. 



Age A pj>, 



The second sUb-objective I had only one exercise. Kesults are 
presented in Table 23 (SSU), The investigator deemed the exercise 

Table 2.3 (Exercise Area SSU) 



Objective: 

I B 


Total // 
Items 


Item Foi'mat 


Administered 


Arc ] 


Leve 


Is 


Cog. Level 


Comp* 




k'Kl k Ch « 


Comb. 


Ind. 


Group 


9 


13 


17 


Ad. 


K 


C 


A An 


S 


E 


1 








1 


1 






3 


1 


J 




1 


t 







[ Ail 


I, \ 


jevel 


// Valid 


if Not Valid 


// Exem. 


// Con. Val. 


Age App. 














1 










1 




1 


1 


J 
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valid, ©xswplary, and appropriate for the ages indicated* 
Sub-Hsbjective I C was not represented by any exercises. 

Objective II. Use Analytic-Scientific Procedures Effectively. 

Only one exercise measured the first sub'-objective, II A. Judges 
classified it as valid. Details are recorded in Table 24 (SSU). 

Table Z4 (Exercise Are a SSU) 



Objective: 



Total 
items 




Item Fortaat 


Administered 


Age Leve 


.s 


Cor. 


Le^ 


/el 


1 


Comp. 


Wri. 


Ml.Ch. 


Comb. 


Ind. i 


Group. 


9, 


13 


17 


Ad 


K 


C 


A 


An, 






1 




1 








1 






1 


1 








1 




A£f. Level 


// Valid 


# Not Valid. 


^ Exem# 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 






















1 




1 


3 


1 



The next sub-objective represented by the exercises is II C. Table 
2$ (SSU) reports that thirteen exercises were dcvelofKid for this objective. 

Table ZS (Exercise Area SSU) 



ERIC 



Objective: 

lie 



Total // 
Items 



13 



Item Format 



Comp. WriljMr.Ch. Comb 



Administered 



Ind. Group 



/O 



Age, 



9 13 



6 



levels 



17 



Ad. 



Coe. Levejl 



K C 



6^ 



A An S E, 



kit* Mvel 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


Age App, 




Ch,, 






















13 




15 


; 3 


JZ 



t 
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The review panel and the major investigator judged all items valid and 
exemplaiy. 

Table 26 (SSU) contains Mi relating to sub-objective II D, 
Tabl e ^6 (Exercise Area SSU ) 



Object iv 

11 D 


e; 


Total // 
items 




Item 


Format 


Administered 


Afte Levels 


Coe . Level 


Comp. 


Wrl. 


Ml.Ch. 


Comb. 


Ind. Grouo. 


9 


13 


17 


Ad 


K 


C 


A 


,Anj 


S 




4 






4 






4 


2 


2 


2 


2 




3 




I 






Afi 


I* 


Level 


// Valid 


# Not Valid. 


Exem. 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 












* 










3 


] 




3 


3 



Pour exercises related to this objective and judges ranked three as valid. 
Objective 11 E data are reported in Table 27 (SSU). Checking the table 
reveals six exercises present for this sub-objective and all determined 

Table V2>7 (Exercise Area SSU) 



ERIC 



Objective: 

11 E 



Total // 
Items 



Comp. 



Item Format 



Wri; 



MI k Ch . 



Comb. 



Administered 



Ind. Group 



Age Levels 



13 



17 



Ad. 



Cog. Level 



K 



C A 



An 



E 







~>.'. — 1 — , , — _ , -, ■ ■ ,, ' , 


Aff> ieyel 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


Age App. 


I^ElCoKilllSH 






















6 




6 


i, 
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valid by the judges. Of some interest is the fact that the panel ranked 
all exercises at the cognitive level of analysis. This represents some- 
thing of a shift from previous exercises \«here the cognitive emphasis ^s 
primarily at the lower levels of the taxonoinjr. 

Objective III. Are sensitive to Creative-Intuitive Methods of 
Explaining the Human Condition 

The first objective in this category having exercises was sub-objective 
III As reported in Table 2B (SSU) two exercises, both valid, related to 
this objective. The investigator and judges also considered the exercises 
to be good exanQ)les for teacher use. 



Table 2?. (Exercise Area SSU) 



Objective: 

III B 



Total // 
items 


Item Format 


Administered 


ARe 


Lcve 


Ls 


Cos 


! * Ley 


;,e3 




Comp. 


Wri. 


Ml.Ch. 


Combe 


Ind. 


Group. 


9 


13. 


17 


Ad, 


K 


C 


A 


An 


S 




Z 




1 


1 




J 


I 


i 


z 


Z 


Z 








2 





Re Rs 



A££« Level 



Va Or Ch 



// Valid 



// Not Va,li<i 



^ Exem. 



// Con. Val. 



Age Ap p.. 



/6 



No exercises existed for objective III C. However, III D did have 
three exercises, all of which the judges ranked as valid and exemplary. 
This information is presented in Table 29 (SSU). 
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Table 2.9 (Exercise Area SSU ) 



Objective: 

111 D 




Total // 
Items 


Item Format 


Administered 


Age ] 


Leve 


Ls 


C 


:og 


, Level 


Comp . 


WriJ 


L*Ml ft Ch • 


Comb. 


Ind. < 


Group 


9 


13 


17 


Ad. 


K 


c 


A An 


S 


E 


3 






3 






3 


2 


2 


2 


Z 




3 






i 


Aff. Level 


// Valid 


if Not Valid 


// Exem. 


// Con. Val. 


Ase App , 


Re 


Rs 


Va 


Or 


Ch 






















3 




3 


3 


3 

, , ■■ - 



Objective TI, Have knowledge Relevant to the Major Ideas and Concerns 
of Social Scientists 

The first sub-objective for which exercises exist is IV B. Only one 
exercise was present^ and the investigator and panel classified it as 
valid and an effective example for teachers. Table 30 reports these data. 



Table 30 (Exercise Are a SSU) 



Objective: 

IV B 


Total 
items 


// 


Item Format 


Administered 


Afte Levels 


Cos.. Lei 


/el 




Comp. 


Wri. 


Ml.Ch. 


Comb. 


Ind. Group. 


9 


13 


17, 


Ad 


K 


C 


A, 


An 




1 




J 






1 








] 


J 




1 








Aff. Level 


// Valid 


y/ Not Valid. 


^ Exem> 


// Con. Val.. 


Age App, 


Re 


Rs 


Va 


Or 


Ch, 






















1 




1 


1 


2 
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Table 31 (SSU) indicates six exercises prepared to measure the next 
sub-objective—IV C. However, the writer and judges listed three of these 
exercises as not valid and poor examples for teachers. Also, the three exer- 
cises Judged as failing to have content validity also were not appropriate 
for the age levels indicated. Reasons for being deemed inappropriate were 
either too easy, too difficult, or not relevant to the age level targeted. 



Table 31 (Exercise Area SSU ) 



Objective: 

IV c 


Total 
Items 


// 


Item Format 


Administered 


Age 1 


Levels 


C0£ 


Level 




Comp. 


Wri;j 


L*Ml%Ch* 


Comb* 


Ind* 


Group 


9 


13 


17 


Ad. 


K 


c 


A 


An 


S 


E 








6 






6 


1 


4 


5 


5 


2 


3 


1 








Afl 


* 


4evel 


// Valid 


// Not Valid 


, // Exem. 


ii Con. Val. 


A^e App, 




Re 


Rs 


Va 


Or 


Ch 






















3 


3 


3 


3 


3 



Sub-objective 17 D was the next objective for which exercises were 
present. As indicated in Table 32 (SSU) nine exercises existed to assess 
this objective. However, only six exercises qualified as having content 
validity or being examples productive for teacher use. It also should be 
noted that there v>a8 a heavy loading of exercises at the knowledge level. 
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Tabl e 3Z (Exercise Are a SSL) ) 



Objective: 

IV D 



Total // 
items 



Comp> 



Item Format 



Wri. 



1 



^a.ch. 



Comb, 



Administered 



Ind> Group ♦ 



A j^e Levels 



7 



13 



17 



Ad 



J 



K 



Cog.. Level 



3 



C A 



An 



Aff. Level 



Re Rs 



Va 



Or Ch 



// Valid 



6> 



// Not Valid 



^ Exem, 



il Con. Val., 



Age App. 



The next sub-objective under objective IV also had numerous exercises 
prepared, with a significant nuober of them judged as not ^ lid. Table 33 
(SSU) reports these data. Note that almost fifty per cent of the items 

Table 33 (Exercise Area SSU) 



Objective: 

IV E 



Total // 
Items 



iz 



Comp. 



Item Format 



Wri: 



1 



•Ml. Oh. 



Comb« 



Administered 



Ind. 



Group 



11 



Age Levels 



13 



9 



17 



S 



Ad. 



K 



Cog. Level 



5& 



k An 



3 



E 



Afl 


> • 


*evel 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


A^e App. 


Re 


Rs 


Va 


Or 


Ch 






















7 


5 


4 




6 



prepared were considered not valid. Also note that there existed a heavy 
loading of exercises at the lower cognitive levels, knowledge and 
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coaprehension. Only fifty per cent of the exercises were of exemplary 
quality, contained cont6;it validity, and were appropriate for the targeted 
ages. 

Data for the last sub-objective under major objective IV is recorded 
in Table 34 (SSU). Analysis of the table reveals eight exercises relating 
to this objective with seven being judged as valid. 



Tabl e 3^ (Exercise Are a SSU ) 



Objective: 

IV F 


Total 
items 


// 


Item Format 


Administered 


Age 


Levels 


Coe. Level 




Comp. 


Wri. 


Ml.Ch. 


Comb. 


Ind. Group. 


9 


13 


17 


Ad 


K 


C 


A 


An 


S 










? 








1 


6 


6 


5 


6 


2 








A££. Level 


// Valid 


it Not Vali^. 


^ Exem. 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 






















7 


1 


7 


6 


7 



Objective IV. Have a Reasoned Commitment to the Values that Sustain a 
Free Society 

Three exercises were designed to measure the first sub-objective of 
this major objective. Table 35 (SSU) reports these data. Review of the 
table reveals that the panel presumed all exercises to have content 
validity. 
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Objective: 

V A 


Total 
Items 




Item Format 


Administered 


Age ] 


Levels 


{ 


ZoR, Level 




Comp. 


Wri.J 


LMl»Ch. 


Comb. 


Ind. ( 


Sroup 


9 


13 


17 


Ad. 




2 


A 


An 

• 


i 




3 






1 


2 




1 


2 


2 


2 


z 


A£i 


c. 


iGvel 


// Valid 


// Not Valid 


. // Exern* 


ii Con. Val. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 
















2 




1 


3 




3 


3 


3 



Three exercisorj were also evident for the next sub-ob jective, V B. 
Table 36 (SSU) lists the pertinent information. From obseirvation, one can 
discern these exercises as valid and exemplary. Of some interest is that 
the review panel considered the exercises to possess affective qualities 
relating to valuing and characterization. 



Table, 3^ (Exercise Are a33U ) 



Objective: 

Vb 



Total // 
items 



Item Format 



Comp 



Wri. 



Ml.Ch. 



Comb . 



Administered 



Ind,. .Gr oup 



A;^e Level s 



9 13 17 Ad 



Cog. Level 



K 



C A 



An 



J 



] 



A££. Level 



Re 



Rs 



Va 

J 



Or 



Ch 



// Valid 



Not Valid. 



# Exem. 



// Con. Val.. 



TASK IT 



Table 37 (SSU) reports ini'ormation relating to the third sub-objective, 
V C, Two exercises, V)oth valid, were present for measuring, this objective. 



Table 37 (Exercise AreaSSU) 



Objective: 

V c 


Total 
Items 


// 


Item Format 


Administered 


Age ] 


Leve: 


Ls 


Cog. Level 




Comp. 


Wri.j 


L Ml % Ch • 


Comb# 


Ind . Group 


9 


13 


17 


Ad. 


K 


C 


A 


An 


S 




z 








z 


z 






i 


Z 


Z 












! 


Afl 


E. 


.evel 


// Valid 


// Not Valid 




// Con* Val# 


Age App. 


Re 


Rs 


Va 


Or 


Ch 




















Z 


2 




z 


z 


2 



The next sub-objective for which unreleased social studies exercises 
were created is V K, Table 38 (SSU) reports two exercises developed and 
determined valid and worthy of emulation. 

Tabl e 33 (Exercise Are a S5U) 

Objective: 



Total // 
items 


Item Format 


Administered 


Arc Levels 


Cor, Level 


Comp« 


Wri. 




Comb« 


Ind« Group. 




,13 


17, 


Ad, 


K 


C„ 


A 


An 


S 




2 




2 






z 




2 
















% 



A£i 




^evel 


// Valid 


// Not Valid. 


^. Exem., 


// Con. Val.. 


* ■ ■ 

Age App. 


Re 


Rs 




Or 


Ch 




















2 


1 




2 


2 


z 
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Only one exercise existed to measure the :Ust oub-objective of Ob- 
jective V, V F, The panel considered the exercise valid and was pleased to 
see that it represented the synthesis level of the taxonony. Table 39 
(SSU) reports the related information. One might also note that the upper 
level of the affective domain was represented by t'rds exercise. 



Table 39 (Exercise Area SSU) 



Objective: 

Yf 


Total 
Items 




Item Format 


Administered 


Age ] 


Levels 


Cog. Level 




Comp. 


Wri.J 


LMl.Ch. 


Comb. 


Ind. 


Group 


9 


13 


17 


Ad. 


K 




N 


An 


t S 


E 


1 








1 


1 






1 


1 


1 










1 




Afi 


• • 


ievel 


// Valid 


// Not Valid 


, // Exem. 


ft Con. Val. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 




















J 


J 






J 


J 



Scmimary of Social Studies Exercises, Released and Unreleased 

The last thirty-nine tables have reported data relating to all social 
studies exercises I both released and unreleased. Table 40 summarizes these 
tables reporting on the total number of items for each major objective as 
well as the number of exercises for each sub-objective. The reader also can 
obtain data relating to the total number of exercises judged to have content 
validity or not to have content validity as well as the number considered 
exemplary. 



TABLE 
SOCIAL STUDIES 
REIiEASED AND UN 



v<»— i 1 m, ,1 - , ■ ., 

Objective 


# of 


Item Format 


Admin. 


Ace Levels 




Cofir. Lev* 






Come 




ML. Ch. 


Comb.. 


Ind. 




q 






Ad 


K 


OlAl 


AN 

Ail 


e 
o 


£ 


Ik 


10 





u 




6 


10 




6 




2 




3 


6 










IB 


2 








2 


2 







0 


0 


o 




1 








... 

1 


IC 


2 




1 




1 


2 


— • 


1 


1 






— 
— 






. 


— 





IIA 


5 





2 


3 




2 


3 


1 


2 


3 


3 




1 




3 






IIB 


0 


^ ... 
































IIC 


26 


8 




14 




9 


— - •• 

17 


9 


15 


12 


10 


11 


8 


4 




1 


1 


IID 


9 






9 






9 


7 


5 


4 


4 


1 


5 




1 

mm 




2 


IIE 


11 




8 


3 




3 


8 


7 


5 




4 




2 




8 








0 ; 














— -- 








— — .. 


— - 






" 




- — - — — 



IIIA 


' " ' 1 
1 




1 








1 




1 


1 


1 


1 












IIIB 


6 




5 


1 




2 




2 


6 


6 

w 


K 
*/ 








L 






' 1 

mc 


0 


































HID 


6 


1 




5 





-• 

— "— — 


6 






✓ 






5 




n 
X 




■ — 


' — 1 ■ - , 

m 


1 






1 






1 




1 


1 


1 




1 










IVB 


4 




2 


2 




1 


3 


1 


1 


2 


2 


L 


2 


1 








IVC 


14 




8 


6 






14 


k 


7 


10 


10 




7 


3 








IVD 


la 




A 


14 




3 


15 




7 


A 




i — ^ 

;i3 


1 


1 








IVE 


27 




3 


24 




L 


23 


12 


18 




20 




14 


- 








IVF 

- - ■ — , - - - ■ . 


19 




1 


18 





1 


18 


/. 


12 


15 


14 




7 


1 






— 


VA 


9 




3 


1 




5 


5 


4 


5 


6 


7 


6 


3 


2 






3 




VB 


11 




1 


2 


8 


5 


6 


1 


4 


11 


10 


r 

i 


1 




1 


1 


4 


VC 


3 








3 


2 


1 




2 


3 


2 


i 
f 












VD 


0 


































VB 


7 




4 




5 


_ 6_ 


1 


3 


D 




2 ;. 








2. 




cr 














^,.2, ... 


1 




2_ 


2 


♦ 








1. 





194 



f ♦ 

AO 

Smm TABLE 
RELEASED EXERCISK5 






4d 




Affec. Level 


# Valid 


# not 




# Con. Val. 


Age App. 


Totals 


Re 


Rs 


Va 


Or 


Ch 




Valid 




3 


3 








9 


1 


10 


10 


10 




1 






1 




1 


1 


1 


1 


1 


[Zeroises 




1 


1 






2 




2 


2 


2 


for Is 14 
'/% 


1 


~« — 








4 


1 


3 


3 


4 












X 


X 












1 








25 


1 


24 


24 


24 




1 










6 


3 


7 


7 


8 










1 




10 


1 


11 


10 


_11 ^ 


•bcercises 
tor lis 51 
26^ 












X 


X 




^ 














1 




I 


1 






1 




2 


1 


6 




6 


6 


6 


Exercises 












X 


X 








for Illj 

113 






1 






6 




6 


6 


6 


75g 












1 




1 


1 


1 














3 


1 


3 


2 


2 




1 










9 


5 


9 


.10 


10 




2 






1 


1 


13 


5 


13 


13 _ 


13_ 




1 


1 




1 




19 


8 


18 


18 


18 ] 


Sxercises 












16 


3 


16 


_ 16 


16 


for IV: 
$3 





— 


5 





3 


9 




9 


9 


9 


43^ 






1 




9 


11 




10 


10 


10 








1 




2 


3 




3 


3 


3 














X 


X 














2 


3 


2 


7 




7 


7 




Sxercioes 










2 


3 




_ J- 


1 3^___ 


3 


for Vt 33 
V7i 










164 


30 


163 






m 
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Elaboration of these data will occur in the discussion section of this 
report. But first one needs to obtain information relating to exercises 
prepared for citizenship, both released and unreleased. 

Citizenship Exercises, Released 

Objective I. Show Concern for the Welfare and Dignity of Others 

The first sub-objective deals with having individuals treat all indi- 
viduals with respect, I, A, Table 41 (CR) indicates five exercises developed 
for this sub-objective and that the review panel judged three exercises as 
possessing content validity. Two were considered not valid or exemplary. 



Tabl e 4-1 (Exercise Are a CR ) 



Objectives 

lA 


Total 
items 


// 


Item Forioat 


Administered 


Arc Levc: 


Lb 


Coe. Level 




Comp. 


Wrl. 


Ml.Ch. 


, Comb • 


Ind. 


Group. 


9 


13 


XX, 


Ad 


K 


c 


A 


An 


S, 




5 






1 


4 


5 




1 


5 


3 


2 


2 








1 




Afi 


u Level 


// Valid 




Wot Valid, 


*.Exem.. 


// Con. Val., 


Age App, 


Re 


Rs 


Va 


Or 


Ch 


















P 




3 


2 


3 


3 


3 



U... — — — 



However, .In some instances, exercises determined not valid for one objective 
were valid and exemplary for another objective. 

No exercises existed for sub-objective I B '♦Consider the consequences 
for others of their own actions" and for sub-objective I C "Guard safety and 
health of others." Sub-objective I D was considered^ and Table 42 (OR) 
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reports the data relating to it. Specifically, this sub-objective relates 
to whether individuals "help ether individuals voluntarily." 



Table 4i2 (Exercise Area C R ) 



Objective: 

Id 


Total 
Items 




Item Format 


Administered 


Age 1 


Levels 


Cog. Level 




Comp. 


Wri;j 


^Ml.Ch. 


Comb. 


Ind. 


Group 


9 


13 


17 


Ad. 


K 


C 


A 


[An 


S 


iE 




3 






1 


4 




J 


3 


2 


3 












I 




Afi 


■ 


iCvel 


it Valid 


// Not Valid 


// Exem. 


1 

j// Con. Val. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 


















1 




J 


J 


/ 


1 


] 



Analysis of the table reveals that of four exercises developed, three were 
not valid. Also notice that the questions cluster at the lowest cognitive 
levels knowledge. 

Sub-objective I E "Are loyal to country, to friends, and to other group© 
whose values they share" is not represented by any exercises. Objective I P 
"Understand and oppose unequal opportunity in the areas of education, 
housing, employment, and recreation" is the next sub-objective for which 
exercises were written. Table 43 (GR) lists related data. Only one 
exercise was present for this objective and the panel menibers determined 
it valid and exemplary. Both cognitive domains existed in this exercise* 
Sub-objective I G "Seek to improve the welfare of groups of people less 
fortunate than they" had no exercises* 
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Tabl efJ (Exercise AreaCR, ) 



Objective: 

If 



Total // 
items 



1 



Comp . 



Item Format 



Wri. 



Ml.Ch. 



Comb. 



3 



Administered 



Ind> Groups 



3 



Age Levels 



9 13 17 ,Ad 



J 



Cog., level 



K 



C 



An 



Aff. 1 


^evel 


// Valid 


// Not Valid 


^ Exem. 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 
















J 






i 




I 


J 


1 



Objective II . Support Rights and Freedoms of All Individuals 

The first sub-objective of objective II was "Understand the value of 
constitutional rights and freedoms, II A. Table kk (OR) reveals the 
related data* 



Tabl e 44 " (Exercise Are a CR ) 



Objective: 

IlA 


Total 
items 




Item Format 


Administered 


„ Arc Levc 


Ls , 


Cofi 


Level 


( 


Comp* 




Ml.Ch. 


C€)«9lb • 


Ind. Group. 


9 


13 


17 


Ad,. 


K 


C 


A 


An 


S 




z 








z 


z 




1 


Z 














1 


A££. Level 


Valid 


» Not Vallfl. 




Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 




















1 


1 


1 

-. 


1 


1 


1 



Review of the table indicates two exercises prepared , one being valid and 



one not valid. However > the valid question did represent the upper levels 
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of both the cognitive and affective levels. 

Sub-objeotive II B, the next sub-objective for which exercises were 
developed, dealt with whether individuals "recognize instances of the 
proper exercise or denial of constitutional rights and liberties, including 
due process of law." As exhibited in Table 45 (OR), NAiSP obtained two 
exercises and panel members judged both valid. 



Tabl e 4-*^ (Exercise Are aCR ) 



Objective: 

n B 


Total 
items 


W 


Item Format 


Administered 


Age Levels 


Coe. Lei 


/el 




Comp. 


Wrl. 


^a.ch. 


, Comb. 


Ind. Group. 


9 


13 


17 


Ad„ 


K 


C 


A 


,An 


S 


a. 


1 




I 




\ 


z 


— • • 


] 


Z 


J 


1 


J 


1 








Af£. Level 


# Valid 


U Not Valid. 


# Exern* 


// Con. Val.. 


Age App. 


Re 


R8 


Va 


Or 
























2 




z 




2 



Sub-objective H C was concerned with whether individuals "defend 
rights and liberties of all kinds of people uniformly." Observation of 
TaWe 46 (CR) indicates two exercises prepared and classified as being 
valid. 
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Table 46 (Exercise Area Q R ) 



Objective: 

ri c 



Total // 
Items 



Item Format 



Compi 



Wri; 



Ml « Ch • 



Comb. 



Administered 



Ind * Group 



lip^e Levels 



9 13 



17 



Ad. 



Co g. Leve l 



K C 



An 



Aff. } 


^evel 


// Valid 


it Not Valid 


// Exem. 


if Con. Val. 


Age App, 














1 




3 


I 


Z 




z 


z 


z 



Objective in. Help Hainbain Law and Order 

Only two of the six sub-objectives of Objective Three were represented 
by exercises. Objective III A "Understand the need for law and order" was 
the first considered and its data are reported in Table 47 (OR), NABP 
used three exercises, and the judges deemed all to be valid and exemplary* 



Table 4-7 (Exercise Are a CR ) 



Objectlvei 

UI A 


Total // 
Items 


Item Format 


Administered 


Age 1 


Levels 


Cog. Leve' 


L 


Comp. 


WrifJ 


[Ml.Ch. 


Comb. 


Ind. < 


Sroup 


9 


13 


17 


Ad. 


K 




A An 


S 


E 






% 




J 


3 




1 


1 


1 


2 




3 








Aff. 


^evel 


i\ Valid 


Not Valid 


// Exem. 


// Con. Val. 




Re 




Va 


Or 


Ch 






















3 




3 


i 


3 
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Those sub-objectives of Objective III not represented by exercises 
were m B "Are conscious of right and wrong behavior," III C "Comply 
with public laws," III D "Help authorities in specific cases," and 
in P "Inform themselves about the law. " 

The next sub-objective having exercises was Objective III E, dealing 
with whether individuals "protest unjust rules openly." Table kB (CR) 
designates only one exercise produced. Judges classified it as valid, 

Table^y (Exercise Area CR ) 



Objective: 

in E 



Total // 
Items 


Item Format 


Administered 


Age ] 


Levels 


Cog 


;. Level 


Comp. 


Wri;j 


LMl.Ch. 


Comb. 


Ind. ( 


Srpup 


9 


13 


17 


Ad. 


K 


c 


A 


An 


S 












1 


] 










1 




1 




• 







Aff 
RefRs 



Va 



evel 



Or 



Ch 



// Valid 



1 



// Not Valid 



// Exem. 



// Con. Val. 



Age App. 



1 



Objective "N* Know the Main Structure and Functions of Our Government 

"*^cognize the purposes of government," IV A is the first sub-objective 
Analysis of Table 49 (CR) reveals that one exercise was present. It was 
r« as vaHd and exemplary» 

Sub-objective IV B "Recognize the main functions and relations of 
governmental bodies" had eight exercises. However, the majority of them, 
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Tabl e 4-9 (Exercise Area CR ) 



Objective: 

lY A 


Total if 
items 


Item Format 


Administered 


_ Age Levels 


Coe. Level 


Comp. 


Wri. 


Ml.Ch. 


Comb* 


Ind. 


Group . 


9 " 


13 


17 


Ad 








1 




1 






1 




1 


1 


I 




1 


1 


1 


1 




A£i 




Level 


Valid 


/A Not Valid. 


^ Exem, 


// Con. Val., 


Age App. 


Re 


Rs 


Va 


Or 


Ch 




* 


















1 




1 


1 


i 



nix, did not possess content validity, nor were they good examples for 
teacher use. Table 50 (CR) reports these data. 

Table $0 (Exercise Area CR ) 



Objective: 

lY 8 



Total // 
Items 



IttiOi Format 

Comp> I Wri ;iMl.Cht l^omb • 



Administered 



Ind> Group 



Age Levels 



9 13 



17 Ad. 



Cog^ Level 



K C A Ani StE 



9 



] 2 



71 



...„„Af f lieyel Valid 


• 

if Not Valid 


, // Exern* 


S Con. Val. 


Age App, 
























z 




1 


z 


z 



The third sub-objective IV C deals >dth whether individuals "recognize 
the imporbance of political opposition and diverse interest groups." 
Table 51 (CR) indicates tws exercises prepared. The review panel determined 
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one exercise valid* 



Tabl e 5l (Exercise Are a CR) 



Objective: 

lY C 



Total // 
items 



Item Format 



Comp. 



Wri. 



1 



^a.ch. 



1 



Administered 



Ind, Group. 



Age Levels 



9 13 17 I Ad 



1 



K 



Cog., Level 



C A 



An 



A£l 


c. I 


^evel 


// Valid 


// Not Valid. 


^ Exem. 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 






















1 


1 


1 


1 


1 



Five exercises existed for sub-objective IV D, "Recognize that 
democracy depends on the alertness and involveinent of its citiisens, and 
know how citizens can affect government." As Table 52 (GR) denotes, four 
of these exercises were valid. Hoviever, the review panel only felt three 

Table 52 (Exercise Area C R ) 



Objective: 

lY D 


Total // 
Items 


Item Format 


Administered 


Arc ] 


Levels 


COf 


L« Level 


Comp. 


Wri;j 


i'Ml%Ch. 


Comb. 


Ind. Group 


9 


13 


17 


Ad. 


K.C 


A 


An 




E 


5 


z 


1 




z 


4 


1 




3 




2 


4 




> 




■ 



A££« Level 



// Valid 



Not Valid 



// Exem. 



2 



// Con. Val. 



Age App < 



3 
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to b9 exmglaxy for teacher use. Here ve have an instance of an exercise 
toving validity, but not being of great value to the teacher* 

The next table, 53 (OK), reports the results for sub-K)bjective IV 
"Recognize the structure and operation of political parties." Tw exer- 
cises, both valid, were prepared; however, only one was exemplary. 



Table 53 (Exercise Are aCR ) 



Ob j ec t ive : 


Total 
items 


// 


Item Format 


Adtrfinistered 


Arc 


Levels 


Cor. 


te> 


/el 


1 


Comp. 


Wri. 


Ml.Ch. 


Comb. 


Ind« Group. 


9 


13 


17 


Ad 


K 


C 


A 


An 


S 




2 


1 






1 


z 






z 


1 


Z 


2 










Aff. Level 


// Valid 


Not Valid. 


^ Exem* 


// Con. Val.. 


Age App. 


Re 




Va 


Or 


Ch 






















z 




1 


1 


i 



No exercises were present for sub-objective IV P, "Know structure of 
school and student government*** 

Objective V. Seek Community Improvement Through Active, Demoeratie 
Participation 

"Believe that each person (s civic behavior is important, and convey 
this belief to othem** and "Recognize important civic problems and favor 
trying to solve them," stib-objectives V A and V B respectively, were not 
represented by any exercises. The third 8ub>>objeetive of Objective V is 
the first for which exercises are present. This sub-objective, V G, 
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considers whether persons tested "actively wojHk for commimity inqsrovoraent." 
Table 54 (OR) manifests the data* NABP had seven items, five of which 
achieved content validity and possessed model qualities. It is of some 
interest that exercises tested for the upper levels of the affective domain. 
This is partly due to the nature of the objective. If people do woric for 
community improvement they must demonstrate that they value certain things 
and have incorporated these values into their behavior network. 



Table 5 4 (Exercise Area C R ) 



Objective: 

YC 


Total 
Items 




Item Format 


Administered 


Age ] 


Leve 


Is 


Cog. Level 




Comp. 


wri;j 




Comb. 


Ind. Group 


,9 


13 


17 


Ad. 


K 


C 


A Ani 


S 


E 


7 






1 




7 




J 


1 


2 


5 








1 




Afi 


> • 


ievel 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 


















I 


5 


5 


2 


5 


5 


5 



The fourth sub-objective^ V D is designed to assess whether persons 
<*participate in local> state and national governmental processes," Only one 
item was administered for this sub*^bjective, and it ranked as valid and 
exenplary. Table 55 (CR) repoHs the pertinent data. 
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Tabl e 55 (Exercise Are a CR ) 



Objective: 

Vd 



Total 
items 




Item Format 


Administered 


Arc 


Leve. 


Ls 


< 




Le^ 


;el 


( 


Somp* 


Wri. 


Ml.Ch. 


Gcmb. 


Ind. ( 


3roup« 


9 


13 


17 


Ad 


K 


C 


A 


An 


S 




1 








1 


1 








1 


1 












Aff. Level 


il Valid 


it Not Val 




# Exem. 


il Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 






















1 


1 




1 


1 


1 



''Apply doBoeratic procedures on a practical level t<hen working in a 
groi^),** V B| is the next 8vib->ob3eetive under Objective V* RevieK of Table 
56 (CR) reveals that three itens esdsted for this objective, all of which 
had content validity and exeiqplary qualities* 

Tabl e 5^ (Exercise Ara a CR ) 



Objective: 

VE 


Total 
Items 




Item Format 


Administered 


Age ] 


Levels 


Cog. Level 




Comp. 


Wri;j 




Comb. 


Ind, Group 


9 


13 


17 


Ad. 


K 


C 


lA 


lAni 


s 


E 


3 






1 


2 




3 




1 


1 








1 


2 






Aff. 


^evel 


a Valid 


Not Valid 


// £xem« 


// Con. Val. 


Age App. 


Re 




Va 


Or 


Ch 




















1 


3 




3 


i 


3 
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Sub-objective V P "Display fairness and good sportsmanship toward 
others" had no exercises. 

Objective VX. Understand Problems of International Relations 



"Are aware of the problems of international conflict and dangers to 
national security" is sub-objective VI A. Table 57 (GH) designates the 
data relating to this sub-objective. Study of the table indicates five 
exercises developed with four judged as valid and exeniplary. Both the 
cognitive and affective levels are represented. 

Tabl e SI (Exercise Are a Cfi ) 



Objective: 

VI A 



Total # 
items 


Item Format 


Administered 


L Afte 


Levels 1 


Coe.t Level 


Coso). 


Wri. 


Ml.Ch. 


Comb. 


Ind. 


Group. 


9 




17, 


A4 




-1 
2 




5 




5 






5 




3 


3 


2 


z 


1 


2 







Afi 


I, 1 


Level 


y/ Valid 


U Not Valid. 


^ Exeitt« 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 




















1 


4 


1 


4- 


4 


4 





Stib-objective VI B "Seek world peace and freedom for all peoples" did 
not have any exercises. 



Objective VII. S>:q^port Rationality in Conuunicationi Thought and Action 
on Social ProbleuB 



The first sub«objective| VII A, deals with assessing whether individuals 
"txy to inform themselves on socially important matters and to understand 



r 
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alternative viewpoints. Information relating to exercises assessing this 
objective Is denoted in Table 5d (GR)« Exercises here i«ere predominately 
poor %fith only one Item classified as valid and five being in the non-valid 
categoxy. 

Table 5$ (Exercise Area CW ) 



Objective: 

VI] A 


Total iSf 
Items 


Item Format 


Administered 


Age 1 


Leve 


Ls 


( 


:;og. Level 


Comp. 


Wri;j 


/Ml»Ch* 


Comb« 


Ind. Group 


9 


13 


17 


Ad. 


K 


CjA 


An 


S 


E 






3 


3 




6 




1 


3 


4 


5 


2 


2| 

j 


2 







Aff. ievel^ 
[Ch 



1 



// Valid 



1 



// Not Valid 



// Exem. 



if Con. Val. 



1 



Age App. 



1 



Sub-objective VXI B considered whether menibers of the targeted popula« 
tlons ^'evaluated eonnmnlcatlons critically and form their ovn opinions 
independently." Table 59 (CR) Indicates one exercise produced for this 
objeetlve, and classified as valid and exenqplaiy. 

Tabl e 59 (Exercise Are a C ) 



ERIC 



Objective: 

Vll B 



Total 9 
items 


Item Format 


Administered 


Arc 


[iCvels 


<?0P 


L. lei 


;el 




Comp. 


Wrl. 


^a.ch. 


Comb. 


Ind« Group. 


9 




17 


Ad,, 


K 


C 


A, 




S 




1 








1 


1 




1 


1 








1 









Aff. Level 



Valid 



1 



// Not Va , 114 



1 



// Con. Val., 



1 



1 
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Bxiercises lor Objective VH C "Weight alternatives and consequences 
careful3y, then make decisions and carry them out without undue delay" were 
not present. 

Data relating to Objective VII D "See relations among social problen» 
and have good ideas for solutions" are recorded in Table 60 (CR). The 
table connotes two exercises made with only one possessing content validity 
and exen^lary qualities. 

Table 60 (Exercise Area CR ) 



Objective: 

VII D 



Total 0 
items 



Item Format 



Comp. 



Wrt,. 



Ml.Ch. 



Comb. 



Administered 



Ind. Group. 



Afie 



Leve 



9 13 



17 



Ad 



K 



Cog.. Level 



An 



1 



A£l 


l» Level 


// Valid 


a Not Valid. 


^ Exexn. 


// Con. Val.. 




Re 


Rs 


Va 


Or 


Ch 






















1 


1 


1 


] 


1 



Again^ no exercises existed for Objective VII E» "Support free Gosmuni- 
cation and communicate honestly with others." 

The last sub-objective of this division, VII P, relates to whether indi- 
viduals in the sample "understand the role of education in developing good 
citizens." Table 61 (CR) reports that only one exercise was developed and 
that this was not considered valid. 
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Tabl e fel (Exercise Area OR ) 



Objecti*/ 

vn F 


'tis 

N 


Total 
Items 




Item Format 


Administered 


Age ] 


Levels 


Cog. Level 




Comp. 


Wri;j 


L'Ml*Ch* 


Comb. 


Ind. Group 


9 


13 


17 


Ad. 


k C 


A An 


S 


E 


1 




1 






1 




1 


1 


1 


1 




1 










Afi 




4evel 


// Valid 


// Not Valid 


// Exem. 


// Con, Vai. 


Age App. 




Re 


Rs 


Va 


Or 


Ch 
























1 









ObJecti5.ve VIII. Take RftsponsibiUty for Ovm Personal Development and 

Obligations 

Exorcises existed for only one sub-objective under Objective VIII, 
These exercises related to whether persons "further their own self-improve- 
ment and education/ VIII A. Table 62 (CR) discloses four exercises ad- 
ministered and of this total, two achieved content validity and exemplary 
qualities. Both the cognitive and affective domains were represented! but 
the cognitive level centered on the lowest level, knowledge. 

It is of some interest to note that the majority of the sub-objectives 
had no exercises t siib-objectiva VIII B *<Plan ahead for major life changes 
Vm C "Are conscientious) dependable , self -disciplined, and value excellence 
and initiative," and VIII D "Economically support self and dependents." 
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Table ^ ' j (Exc)rcise Are aCR ) 



Objective: 

YlU A 



Total it 
items 



Item Format 



Compi 



Wri. 



Ml.Ch. 



1 



Combt 



Administered 



Ind# Group ♦ 



4 



Age Levels | Cog . Level 



13 



17 



Ad 



Aff. Level 



Re Rs 



Va 

1 



9t 

1 



Ch 



# Valid 



// Hot Valid 



^ Exem. 



// Con. Val., 



Age App. 



Objective IZ. Help and Respect Their Own Families 

This objective vfas designed for two population groups: a group for 
ages 9, 13, and 17; and a group for adults. The first sub-objective for 
the first age groi:5>ing was "Respect the reasonable authority of the par^ 
ents, or guardians, and help with home duties and problems," DC A, The 
corresponding sub-objective IX A for the adult group was that individuals 
"provide for the basic needs and health of their children." Both objectives 
are considered in Table 63 (CR). The table designates that three items mve 

Table ^3 (Exercise AreaCR ) 



ERIC 



Objective: 

IX A 



Total // 
Items 


Item Format 


Administered 


Age ] 


Leve 


Ls 


.CP8 


L. Level 


Comp. 


Wri;] 


.'Ml * Ch • 


Comb* 


Ind. 


Group 


9 


13 


17 


Ad. 


K 


C 


A 


An 


s 


E 


3 


3 








3 




1 


1 




1 


1 






L... 







z 



// Valid 



// Net Valid 



// Exem. 



// Con. Val. 



A^e App. 



f 
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prepared. The two items for the first age group were valid while the 
exercise for the adult population was not. Neither was the exercise for the 
adult population considered exemplary, 

Sub^bjective B for Objective IX also was designed for two age group- 
ings. For the age grouping 9, 13, and 17, the objective was "help younger 
brothers and sisters to develop irAo good citizens," while the corresponding 
objective for adults was "encourage cooperative, ethical relations to au- 
thority and to other individuals." Table 64 (CR) reports these data. Only the 
exercise for age 9 was valid. The panel members determined both the item 
for age 13 and for adults as not meeting the thrust of the sub-objective. 

Tabl e 6 Ar (Exercise AreaCR) 



Objectiv 

IX E 


e: 

; 


Total // 
items 


Comp. 


Item 
Wri. 


Format 


Comb* 


Administered 






Leve 


Is 




P,?i 


L;.., 


Lei 


^el 


3 








3 


Ind. Group. 

3 


9 
1 


13 

1 


17 


Ad 

1 


K 
1 


C 


A 


An 


S 




Re 


Aff. Level 


il Valid 


. // Not Valid. 


^ Exem* 


// Con. Val., 


Age App. 




















1 




1 


z 


1 


1 


1 



Citlxaoship Bboercises, Unreleased 

The unreleased citizenship exercises relate to the same objectives, 
no these objectives wiU be referred to by the outUne identifications 
rather than writing thea out in detail. 
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Objective I. Show Concern for the Welfare and Dignity of Others 

Table 65 (CU) lists that of four items prepared for sub-objective I A, 
three were valid and exemplary. 

Table ^ (Exercise Area CU ) 



Objectiv 

I A 


e: 


Total // 
items 




Item 


Format 


Administered 


Age Levels 


Coe. Level 


Comp. 


Wri. 


Ml.Ch. 


Comb. 


Ind. GrouD. 


9 


13 


17 


Ad 


K 


C 


A 


Axi 


S 


.a. 


4 




1 


3 










4 


3 


3 




2 










Afl 


> • 


Level 


* Valid 


// Not Valid. 


^ Exem. 


// Con. vTal.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 


















2 




3 


1 


3 


3 


3 



Table 66 (CU) exhibits the data relating to sub-objective I C. As in the 



Table 6^ (Exercise AreaCU ) 



Objective: 

Ic 



Total // 
Items 



Item Format 

Comp . I Wr 1 .j Ml * Ch . I Comb . 



Administered 



Ind ♦ Group 



Age Levels 



9 13 



17 Ad. 



C p g « ^ ^ Level 



K C 



A Ant S E 



1 



Aff. I 


4evel 


Valid 


// Not Valid 


// Exem. 


It Con. Val. 


Age App. 




















1 






1 


3 


1 


z 


z 
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previous case^ four exercises existed vdth three being judged valid. Hof^> 
ever, the investigator and panel members considered only two as good exanqples 
for teacher use and appropriate for the intended ages. 

The next objective, I had three exercises. Only one was judged 
valid and exemplary. Table 67 (CU) reports these data* 

Table 67 (Exercise Are a CU ) 



Objective: 


Total 
items 




Item Format 


Administered 


Akc Levels 


Cofl 


Level 


( 


Comp. 


Wri. 


^a.ch. 


Comb. 


Ind. Group. 


9 


13 


17 


Ad 


K 


C 


A 


An 




3 


z 






1 


3 




1 


3 




1 




Z 








A££. Level 


// Valid 


y/ Not Valid. 




// Con. Val.. 

"' ■* ' ' 


Age App. 


Re 


R8 


Va 


Or| 


Cb , 
















1 






1 


z 


1 


1 


3 



The final sub-objective in the Objective I category had one exanqsle* As 
revealed in Table 6d (CU), it did not rank as having content validity* 

Table (Exercise Area CU ) 



Objective: 

1 G 




Total 
Items 


// 


Item Format 


Administered 


Age ] 


Leve 


Ls 


Cofi 


,. Level 




Comp. 


Wri;, 


i'Ml.Ch. 


Comb. 


Ind. Group 


9 


13 


17 


Ad. 


I 


C 


A An 






1 


1 








1 






1 


1 


1 








Afi 


Level 


// Valid _ 


# Wot Valid 


, // Exem* 


// Con. Val. 


Age App. 


iir^T^cnt*nisi 












1 


1 


1 


n 




1 
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Objective II, Supijort Rights and Kroedomn of AU Individuals 

Table 69 (CU) reports the data relating to the exercises for the first 
3Ub«objective II A. The table connotes only one exercise generated, but 
it was deemed valid by panel members. 

Tabl e 69 (Exercise Area CU ) 



Objective: — 

11 A 


Total // 
Items 


Comp. 


Item 
Wri. 


Format 
Ml.Ch. 


Comb. 


Administered 

T_ J a-i 






Leve 


Is 




Qof 


5,.' 


Lei 


/el 


1 








1 




uroup* 

1 


9 


13 


17 

1 


Ad 

1 


K 

1 


C 


lA 


An 


s 






Afi 


* 


Lev< 


il 


// Valid 


jLm vaiidL 


^ Exem. 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 


















1 




1 




1 


1 


1 



The next sub«objective II B, had more exercises, four in number, but 
panel rombers judged only tm as having content validity. Table 70 (CU) 
presents these data. 

Table 70 (Exercise Area CU ) 



ERIC 



Objective: 

11 B 



Total // 
Items 



t'omp* 



Item Format 



WriUWl.Ch. 



1 



Comb. 



Administered 



Ind« Group 



1 



Age. 



J 



13 



•evels 



17 



Ad. 



K 



Cog. Level 



An 







-L.!.-,' 




Ati 




[<ev< 


il 


// Valid 


// Not Valid 


// Exem. 


*' Con. Val. 


Age App« 


Re 


Rs 


Va 


Or 


Ch 






















2 


2 


2 


2 


2 
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Objective III. Help Maintain Law and Order 

The first sub-objective, in A, had only one item. As evidenced in 
Table 71 (CU), the panel members considered it to have content validity. 



Tabl e 71 (Exercise Are aCU ) 



Objective: 

ill A 


Total 
items 




Item Format 


Administered 


. Age Levels 


Cos 


;.. Level 




Comp. 


Wri. 


Ml*Ch. 


Comb* 


Ind. Group. 


9 


13 


17 


Ad 


K 


C 


A 


An] 


S 




1 




1 






1 








J 


1 




] 










• • 


Level 


// Valid 


// Not Valid. 


# Exem. 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or| 


Ch 






















1 




J 


1 


1 



Stib-objective III B had two exercises, and as indicated in Table 72 
(CU), both had content validity. Note that both the cognitive and affective 
domains are present* 

Tabl e 72 (Exercise AreaCU ) 



ERIC 



Objective: 

III B 


Total // 
Items 


Item Format 


Administered 


Age 1 


Leve 


Ls 


( 


:;og. Level 


Comp. 


Wri; J 


IMl.Ch. 


Comb* 


Ind. Group 


9 


13 


17 


Ad. 


K 


C 


A An 


|S 


E 


z 




Z 






2 




1 




1 


1 




1 




1 





Aff. 1 


.evel // Valid 


// Not Valid 


// Exem. 


if Con* Vali 


A^e App. 




















1 




2 




z 


z 


z 
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The last sub-objective treated under this division was XU F, As 
denoted in table 73 (CU), only one item, deemed to have content validity, 
was developed. However, there was oome question as to whether this repre- 
sonted sophistication sufficient to challenge the adult population for 
vdiom the exorcise was intended. 



Tabl e 73 (Exercise Are a CU ) 



Objective: 

in F 


Total 
items 




Item Format 


Administered 


Aee Levels 


Cofi 


Level 




Comp. 


LWri. 


Ml.Ch. 


Comib* 


Ind. Group. 


9 


13 


17 


Ad 


K 


C 


A 


An 


,S 


.EL 


1 






1 














J 




J 










Ati 




Level 


if Valid 


# Not Valid. 




// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 






















1 




1 


J 





Objective IV. Know the Main Structure and Functions of our Government 

The first sub-objective considered in the unreleased section was 
XV B, This objective, as revealed in table 74 (C^O, was one of the few 
havin/» numerous exercises developed for it, fifteen in aJ.l. However, 
quantity is not the only factor of note, for bher.o exorcises had over 
fifty per cent of their number, ei/»ht In all, being considered as invalid* 
A3JS0, the investigator and jtanel members considorod only five exercises 
of sufficient quality as to warrant emulation by teachers in their 
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•xercise developjnent. This is perhaps the largont concentration of poorly 
developed exercises* 

TableJ^t (Exercise Are a CU ) 



Objective: 

IV B 


Total 
Items 




Item Format 


Administered 


Ase ] 


Levels 


Cog. Level 




Comp. 


wri;j 


[Ml.Ch. 


Comb. 


Ind, Group 


9 


13 


17 


Ad. 


K 


C 


|A An 


tS 


E 


15 


4 


3 


? 




7 




5 


7 


6 


7 


10 


4 


] 






Afj 


c. 


ievel 


// Valid 


// Not Valid 


// Exetn. 


// Con. Val. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 






















1 


f 


5 


7 


6 



Gub-objectlve IV C data are reported in Table 75 (Gll), Of four exor- 
cises prepared, panel members indicated that three were valid. However, 
only two were considered exemplary and appropriate for the hp^g intended. 

Tabl e 75 (Exercise Area CD ) 



Objective: 

IV c 



Total It 
items 




Item 


Format 


Administered 


Arc Levels 


Coe, Le^ 


/el 


Comp* 


Wri. 


Ml.Ch. 


Comb« 


Ind* GrouD. 


9 


13 


17 


Ad 


K 


C 


A 


An 


S 




4 


1 


1 


2 






4 




j2 


V 















Aff. Level 



Re 





1 


1 


1 


■ 



' Valid 



// Not Valid 



Exfim . 



// Con. Val., 



Age App. 
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The last sub-objective covered in this division whs TV D. There were 
nine exercises prepared for this objective, but again we have a situation 
in which over fifty per cent of the exercises are not valid. Additionally, 
only three exercises qualified as exemplary, being arpropriate to the 
situation implied, and appropriate for the a/^ea intended. It should be 
noted that items considered as inappropriate could be either too easy, too 
difficult or not relevant to the concerns of the tarf;eted population. 
Table 76 (CU) reports these data. 

Tabl e 7^ (Exercise Area C (J ) 



Objective: 

IV D 



Total 
Items 



Comp. 



Item Format 



Wri: 



•Ml.Ch. 



1 



Comb* 



Administered 



Ind ♦ Group 



1 



Age. 



13 



Re 



Aff. level 



Rs 



Va 



Or 



1 



Ch 



.evels 



17 



Ad. 



6 



Coe« Level 



KIC 



24 



An 



1 



// Valid 



// Not Valid 



j/ Exem. 



// Con. Val. 



Age App, 



Objective V. Seek Community Improvement Thron^h Active, Democratic 
Participation 



Sub-objective V B is the first objective for which exercises w«re 
created. Table 77 (CIJ) discloses threo exerciines produced with two 
considered valid and exemplary. 



BEST cow fflUBUBijE 

Tabl e 77 (Exercise Are a CD ) 



Objective: 

Vb 


Total it 
items 


Item Format 


Administered 


Ase Levels 


Coe. Level 


Comp. 


Wrl. 


m.ch. 


Comb. 


Ind* Group. 


9 


13 


17 


Ad 


K 


C 


A 


An 


S 




3 




1 


Z 




1 


z 






Z 


3 




1 




i 


1 


Afi 


• 


Level 


« Valid 


# Not Valid. 


^ Exem* 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 




















1 


2 


1 


z 


Z 


z 



Table 7d (C1J) reports the data reJatinr; to oub-ohjcctivft V G. Of the 
three exercises prepared, the panel considered al 1 to be vaj.id, however 
only two were perceived as exemplars for teachers ' use« 

Table^TlS (Exercise Are a CD ) 



Objective: 

Vc 


Total 
Items 


// 


Item Format 


Administered 


Age ] 


Levels 


COE 


L. Level 




Comp. 


Wriii 




Comb* 


Ind. Group 


9 


13 


17 


Ad. 


K 




[A 


[An 




LI. 

! 


3 








3 


2 


1 






3 


3 


1 


z 






Afi 


I, 


ievel 


// Valid 


/r' Not Valid 


// Exem* 


// Con. Val. 


Age App, 


Re 


Rs 


Va 


Or 


Ch 








1 ■ 












2 


3 




z 


3 


3 
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Objective VI, Understand Problems of International Relatione 

There viere four exercises prepared for this objective. The exercises 
were not identified as to which of tw sub-objectives was being considered 
therefore, these exercises, as reported in Table 79 (CU), represent a 
collapse of sub-objective categories. Of four exercises prepared, all 
qualified as valid. 

Tabl e 79 (Exercise Are a CU ) 



Objective: 

YI 



Total 0 I 
Items 



Item Format 



Wrl. 



Ml.Ch. 



1 



Cottlbe 



1 



Administered 



Inde Group 



1 



Age 



Levels 



11 



17 



Ad 



K 



Cog. level 



1 



1 




A£f. Uvel 


// Valid 


...am vsui 


# Exem. 


// Con; Val.. 


Age Apr. 


Re 


Rs 


Va 


Or 


Ch 


















Z 




4 






4 


4- 



Objective VII. Support Rationality in Conmmnication, Thought, and 
Action on Social Problem 

Sub-objective VII A had six exercises prepared as denoted in Table 
BO (CU), Observation of the table reveals that only two exercises quali- 
fied as having cotjrf ^nt validity. 
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Table JO, (Exercise AreaCU ) 



Objectiv 

Vll 


' — - ' — j 

A 


Total // 
Items 


Comp. 


Item 
WriiJ 


Format 
.Ml.Ch. 


Comb* 


Administered 




^?e ■ 


Leve 


Is 




Level 


<b 




2 


3 


1 


3 


3 


9 

3 


13 
3 


17 
3 


Ad. 

3 


1 


3 


A An 




lEj 




Afj 




4ev< 


il 


// Valid 


// Not Valid 


// Exem. 


// Con. Val. 


A^e App. 


— . 


Re 


Rs 


Va 


Or 


Ch 






















z 


4 


z 


z 


2 



Only one exercise was present to measure sub-objective VII B, 
Table 81 (Cll) -notes the pertinent data. The panel members did consider 
the sole exercise as having content validity and bein.- exemplary. 

Tabl e gl (Exercise Are aCU ) 



Objective: 

ViiB 



Total a 
Items 



Comp, 



Item F ormat 
Wri. 



Ml.Ch. 



Comb* 



Administered 



Ind, Group. 



Age Levels 



L. 13 17 Ad 



£2& 



K C A An S 



Level 



1 



1 



1 



1 



) 



Re 



Aff. Level i 



// Valid 



1 



// Not Valid 



^ Exem . 



J 



// Con.. Val.. 



Age App. 



3 
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Sub-objective VI 1 G also had only one ewrciac, also va.l.id. Table 
62 (CU) presents the information. 

Table (Exercise Area CU ) 



Objective: 

Yll C 


Total 
Items 




Item Format 


Administered 


Age 1 


Levels 


Cog 


;. Level 




Comp. 


WrilJ 


/Ml ft Ch • 


Comb. 


Ind. Group 


9 


13 


17 Ud. 


K 


C 


A An! 


1 S 


E 


1 




1 






1 




I 


1 




1 


• 








» • 


ievel 


# Valid 


Not Valid 


// Exem* 


if Con. Val. 


A^e App. 


Re 


Rs 


Va 


Or 


Ch 






















1 




1 


J 


1 



Continuing the one-item trend. Table 83 (ClJ) reports the data relating 
to sub-objective VII D, The sole item \*as valid. 



Tabl e g3 (Exercise Area CU ) 



Objective: 

Vll D 


Total 9 
items 




Item 


Format 


Administered 


Ase Leve 


Is 


Coe, Level 


Comp. 


Wrl. 


Ml.Ch. 


Comb* 


Ind* Group* 


9 


13 


17 


Ad 


K 


C 


A 


An 


,S 




1 




1 






1 






1 






1 










A£l 




Level 


// Valid 


it Not Valid. 


f Exem* 


// Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 






















1 






J 


1 
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The final sub-objective for VII, ob.joctive VI J 1% also had only one 
exorcise. Panel judges detenrined that it was valid. Table Bk (CD) 
discloses the data. 



Tabl e g4 (Exercise Are a CU ) 



Objective: 

Vll F 


Total 
items 




Item 


Format 


Administered 


.. Af>e Levels 


Coe. Level 


Comp. 


Wri. 


Ml.Ch. 


Comb* 


Ind« Grouo. 


9 


13 


17 


Ad 


K 


C 


A 


An 


S 




1 




1 










J 


J 








1 










Afi 




Level 


// Valid 


y/ Not Valid. 


^ Exem. 


It Con. Val.. 


Age App. 


Re 


Rs 


Va 


Or 


Ch 






















1 






1 


1 


IC..., ' ' 





Objective Vin. Take Responsibility for Own Penjonal Development and Obligations 

Sub-objective VIII A had four items. Observation of Table 85 (CU) re- 
veals that mnel judges determined ail exercises valid and exomplaiy. 



Table g5 (Exercise Are a CU ) 



Objective: 

Yiil A 


Total 
Items 


// 


Item Format 


Administered 


Age ] 


Levels 


Cop 


;« Level 




Comp. 


wri;j 


;M1 « Ch • 


Comb. 


tnd • Group 


9 


13 


17 


Ad. 


K 


c, 


A An 




£ 


4 








4 




Z 




1 


1 




J 






—J 


A£f. 


^evel 


Valid 


// Not Valid 


, if Exem« 


// Con. Val. 






Va 


Or 


Ch 
















Z 


1 


1 


4 




4 


4 


4 
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Sub-objective VJII C was the noxt. ob.jootivo repronented by exercises. 
Table 86 (CU) reports that only one ihom wfin (ieveiopod for ihitj objective. 
Panel judges considered it valid and exemplary. 

Tabl e S(d (Exercise Area. CU) 



Objective: 

vni c 



Total 0 
items 



1 



Item Format 



C omP( 



Wri. 



Ml.Ch. 



Comb. 



1 



Administered 



Ind» Group* 



13 



1 



17 



Ad 



An 



Age Levels t Co«l* Level 

1 



Aff. I 


jcvel 


// Valid 


Not Valid. 




// Con. Val.. 




Re 
























1 




1 


1 


1 



Objc^Aive IX. Help and Respect Their Ovm Kand l.ios (A^es 9, 13, 17) 

Nature the Development of Their Children as Future Citiaens (Adult) 

As indicated previously, this objective had two forms geared to two 

age groupings. Two exercises were synthesized for sub-objective A for adults 

as indicated in Table 87 (CU), 

Table g 7 (Exercise Area CU ) 



ERIC 



Objective: 

rx A 



Total 
Items 



Item Format 
WriilMl„Ch. 



1 



Comb« 



1 



Administered 



Ind* Group 



A^e Levels 



9 13 



Cog. Level 



17 



A£f « Level 


Valid 


// Not Valid 


)7 Exem. 


// Con. Val« 


















1 






T 




2 
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Panel raenibers agreed that both exercises were not valid. 

Two exercises also were prepared for the 9, 33, 17 age g?;wy.p but these 
exercises were geared to sub-objective IX B» Again the judgeo placed the 
exercises ii^ the not valid category. These data are evident in TaV^e 88 (CU), 

Table BB (Exercise Area CD ) 



Objective: 



Total # 1 


Item Format 


1 Administered 


AftG 


Levels 




Level 


items 
















J- 


K 


2 








2 


2 




1 


1 






1 











A£f« Level 



1 



if Valid 



0 Not Valid 



^ Exem. 



// Con. Val.. 



Age App, 



Summary of Citiaenshi p Exercises > Released and Unreleased 

This last section dealt with exercises designed to measure whether 
individuals at several age levels had attained some level of understanding 
and comadtmenb to various objectives within the domain of citizenship. Table 
89 reports a summary of data relating to all of the citizenship objectives. 
Skimming the table allows one to see which objectives were stressed, which 
were ignored regarding exercise representation, and the content validity of 
these exercises. Other infohaation stressed in the previous citizenship 
tables also is summariased. Detailed discussion of this table and some impU- 
cations and considerations one needs to attend to are treated in some depth in 
the discussion section of this report. 
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CmZBNSHIP 
HELEASED AND 



Objective 






Iten 

WW.. 


I Format 

Ml flh 




-I 

A( 




i 


^56 


Levi 


3l8 










lA 


9 




1 


i U. • Mil « 




JLilCl* 
7 


Group 

o 


.9 


.13 

Q 

7 


0 


Ad 

5 


K 
2 


c 

2 


1 


MM 

■ 


H 


]e 


. B 


0 


































IC 


4 


1 






3 


3 


1 




1 




2 


2 












. ID 


7 


5 






2 


7 




2 


6 


2 


4 


3 


2 










^ IB 


0 
































IP 


1 








1 


1 










1 














IG 


1 








1 


1 








1 


1 














TT A 
UK 


3 






r 


3 


2 


1 


1 


2 


1 


1 


1 








1 




m 


6 




2 


3 


1 


5 


1 


2 


5 


3 


4 


3 


2 


1 








lie 


2 








2 


2 








1 


1 




2 










IIIA 


4 




3 




1 


4 




1 


1 


2 


3 




4 










IIB 


2 




2 






2 




1 




1 


1 




1 






1 




IIIC 


0 



































HID 


0 






— — 




























HIE 


1 








1 


1 










1 


1 










4 


IIIF 


1 






1 




1 










1 




1 










m 


1 




1 






1 




1 


1 


1 






1 








i 


IVB 


23 


, 7 


4 


10 


2 


15 


B 




12 


m .a 

12 


10 


17 


5 




1 




■ "1 * 


IVC 


6 


1 


2 


3 




2 


4 




3 


5 


6 


2 


4 










IVD 


U 


2 


6 


1 


5 


12 


2 


2 


10 


9 


8 


6 


4 




1 




IVB 


2 


1 






1 


2 






A. 


1 


2 


2 












m 


0 
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Ive Level 
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# not Valid 
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Va 


Or 
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0 


3 
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6 














Y 
A 


Y 
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3 


1 


2 


2 


2 






1 
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5 


2 


2 
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X 


X 
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1 


1 
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1 


1 


2 


1 


2 


2 
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4 


2 


4 


4 


4 
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1 
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2 


2 
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4 


4 
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2 


2 
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X 
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■1 
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1 


1 


1 
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1 


1 
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1 




1 


1 


1 


• 
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9 


14 


7 


7 
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4 


2 


3 


3 
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d 


6 


6 


6 
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1 


1 
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j Objective 


# of 




Item Format 


' Admin. 


Age levels , Coe. Lev. 
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. Ind*.. 
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h 1 AM 
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1 
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1 
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1 
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1 


I 
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« 

I 


9 


9 


1 


1 


4 

1 

1 


5 


a 


1 
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1 - ^ 
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1 
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1 
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1 


1 
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0 
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« 

f 


1 


1 


8 


1 


5 


5 


4 


4 


2 


4 
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2 
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0 ^ 























VIIA 


12 




5 


3 


1 


9 


3 


4 


6 


7 


d 


3 
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2 






VIIB 


2 








2 


1 


1 


2 
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1 
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1 
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3 
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1 


2 


1 


1 
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0 
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2 




2 


2 


1 
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VIIIA 
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1 


7 
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4 


1 


2 


2 


3 
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— 


^^^^^^^^---^ 


VIIIB 


0 
































1 
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1 








1 
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1 










VIIID 


0 
































-1 


HA 


5 
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1 


1 
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1 
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3 
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' 
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5 








5 
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2 
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1 


2 


1 
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UC 


0 
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1 
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Reliability of Exercises 

In considering the content validity of exercises, that is, their 
representativeness of the ideal population measuring a behavior(s) or in- 
fonnation, one also contemplates the issue of reliability. Reliability 
can be approached in three ways 5 first from a stance of asking questions 
related to stability, dependability, and predictability. A second approach 
is concerned with accuracy while the third approach is concerned vdth random 
or chance error (Kerlinger, p. 430). 

If we define reliability thro^igh error then the more error the greater 
the unreliability and conversely the less error the greater the reliability. 
Of course, anyone designing exercises and teating instruments desires to 
have the greatest reliability, that is, the least amount of error. 

NABP had some special problems in determining reliability and this 
is discussed in some depth in the Task I report. Basically, NAEP did not 
have an instrument to test, but rather, items adjninistered independently. 
Therefore, NAEP did not have any instrument on which they could run reUa- 
bility figu.ves. But they did address the issue of reliability by deter- 
mining the standard error of each exercise. 

This lack of a total reliability for social studies and citizenship 
exercises might put us at a slight disadvantage for we really cannot discuss 
the reUability of the total pool of exercises. But, from observing the 
standard erroi^. of the individual exercises, one can obtain some insight 
into the accuracy dimension. PuHherwore, one needs to remember that the 
content validity also was determined on individual exercises and not the 
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total pool of exercises. Again, those readers wishiar: greater depth of 
discussion shoidd refer to the Task I report, 

Inber-Rater Reliability 

Since the content validity depended upon the judgment of the major 
investigator and three members of a review panel, the question of inter- 
rater reliability is important. The prime concern of all members was the 
content validity of the exercises and whether they could be used as ex- 
amples. Therefore, one is interested in seeing if the four individuals 
involved reacted in similar, if not identical, ways to the exercises. 

At the beginning of the analysis, the major investigator decided 
that all judges would have to agree as to the content validity of the ex- 
ercises. This, of course, required that judges, after their initial inde- 
pendent reactions, had to have an opportunity or opportunities to discuss 
their perceptions and judgments. This opportunity was provided and the in- 
vestigator was most pleased and even surprised that all judges agreed 3*jii- 
tially as to the content validity and exemplary qualities of 95 per cent of 
the excercises. This agreement was on exercises analyzed independently. 
Little difficulty was experienced in attaining consensus with regard to 
the remaining percentage of exercises. Thus, all the exercises as reported 
U\ this document represent total agreement by judges regarding content 
validity and exemplary qualities. But, the reader should realize that this 
final judgment was arrived at collectively, not independently. It is of 
interest that 100 per cent agreement as to which exercises were outstandingly 
good or bad also was present. 
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Iteadera f«8l that this is too good to be true. Ho,vever. one must 
reMBber that the major investigator ws most careful to seleot for the 
review panel individxmls «ho had great expertise in the area of social 
studies education and al^o «ere most competent in the realm of questions. 

Smmrv 

This section considered the results of the social studies and citizen- 
ship exercises, both released and unreleased. Tables reporting the data 
were presented for reader walysis. Since the tables are rather complete, 
there «as no need for detailed prose description of the data. Tables pre- 
sented data relating to each objective and sub-objective for «hich exer- 
cises were written. Individuals wishing detail as to *ich exercises the 
panel members consider as not having content vaUdity are directed to read 
the tables in Appendix A. 



Discussion of rmr-M -aea bv Ma^or- Qblectlvex 

Social Studies Exercises, Released 

In the discussion, the sub-objective categories have been collapsed, 
and therefore comments relate to each of the major objectives, e.g., I, 
n. III, N, and V. HoKBver, general Impressions will be given about all 
Of the exercises relating to their validity, approirtateness and other i». 
portant elements. Points pertinent to the exercises wiU be inserted 
where appropriate. Some of these points may relate to trends as well as 
relationships extant between certain exercises and particular factors 
considered. Table 90, found on page 93, reports sumnaiy data for 
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both released and Tinreleaaed social studies exercises. The reader may wish 
to view this table before reading about the exercises for specific objectives. 

Objective 1. Have Curiosity About Human Affairs 

There were nine exercises for this major objective in the released 
category. The majority of the exercises utilized the combination format and 
relied solely on individual administration. All age levels were represented. 
The majority of the exercises possessed content validity, seven in total, 
and were effective models for teacher use. Both the cognitive and affective 
domains were represented, with the loading on the lower levels of both 
taxonomies: knowledge and comprehension, and/or receiving and responding. 
These cognitive and affe«}tive levels seem appropriate for the objective 
only aims at assessing an individual's basic understanding and willlngneas 
to attend to the realm of human affairs. 

Objective II. Use Analytic-Scientific Procedures Effectively 

There were twenty-seven exercises prepared for this objective. The 
formats represented all except combination. Exercises were about equally 
distributed between individually administered and group administered, and 
all age levels were present. Again, panel judges dotennined the majority 
of items to be valid, with twenty-two in that category. Most were of 
good quality for imitation by teachers. However, the investigator and 
judges considered far fewer items to be apn^opriate for the age levels 
intended. Some items were too easy or difficult for some of the ages and 
vfere appraised to require some reworking for individuals at particular 
ages to handle them. Of special note is that all levels of the cognitive 



TASK II 



66 



taxonomy were represented, but most were at the lower cognitive levels. In 
this subtest the item makers were more concerned with whether individuals 
knew process rather than being able to apply process. Three of the items 
wre considered to have identifiable affective levels ^dth two of the items 
at the bottom affective level. Overall, the judges considered the exercises 
of good quality. 

Objective III. Are Sensitive to Creative-Intuitive Methods of ExDlaLninc 
the Human Condition *^x«.iu^jg 

Exercises in this category numbered eight. Again all formats are 
represented. The majority of the exercises were group administered and 
all ages were represented. The review panel classified all exercises as 
having content validity and being useful as examples for teachers. Both 
the cognitive and affective taxonomies are represented, with a spread of 
levels present but favoring, in terms of numbers, the higher cognitive 
levels. The panel members listed five exercises at the analysis level. 
This emphasis at the analysis level is encouraging for it indicates the 
exercise development concerned with assessing jjp individuals comprehend 
the basic assumptions and interrelationships relating to the use of methods 
and the human condition. 

Objective IV. Have Knowledge Relevant to the Major Ideas and Concerns 
of Social Scientists 

This objective had by far the most exercises, forty-seven in all. 
The majority were multiple choice in format and were group administered, 
and all ages were represent,ed almost equally, mth regard to content 
validity, the panel judged the majority as valid, but a significant number, 



TASK II 



87 



ten in all, were considered not valid. This represented the highest 
proportion of exercises not having content validity among aU objectives 
for the released social studies grouping. With regard to the cognitive and 
affective levels, both domains were represented. Sill the loading was at 
the knowledge and comprehension levels and the receiving level. Yet five 
exercises existed at the application level and three qualified as analysis. 
Two exercises were at the organizarion level in the affective domain. 

The judges discerned a definite trend that exercises designed spe- 
cifically for age 9 were predominantly at the knowledge and/or comprehension 
levels. It appears that the exercise ievelopers worked under the assump- 
tion that young pupils were incapab].e of handling questions at higher 
cognitive levels. Of course, as previously stated, all th^ exercises 
revealed a predominant, emphasis on questions at the lower levels. 

Prom analysis of these exercises it became apparent that one had a 
better chance of having a question have content validity at the lower cogni- 
tive levels. Tersely put, these exercise developers experienced a higher 
failure rate as they attempted to develop high-cognitive level exercises. 

Objective V. f^ave a Reasoned Commitment to the Values that Sustain a 
Free Society 

I 

Twenty-two exercises were geared to this objective. Again, three of 
the four major formats were represented. Most of the exercises ..-ere group 
administered and equally distributed among the ages of 13, 17, and Adult. 
The judges had total consensus that all exercises possessed content valid- 
ity. Of some note is the fact that all exercises involved both cognitive 
and affective objectives and that they represnted the upper levels, analysis, 
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synthesis, and evaluation in the cognitive domainj and valuin^j, organiza- 
tion, and characterization in the affective domain. This type of situation 
did not exist as frequently as the investigator and judges would have liked. 

General Discussion of Released Social Studies Exercises 

The primary point is that the overwheOjidng majority of exercises 
possess content validity and are appropriate for use as examples by teachers. 
Hot*ever, it is of some concern that the exercises largely stress the lower 
cognitive domains, knowledge and comprehension. Furthermore, it is clearly 
evident that the lower the age level for the exercise, the lower the cogni- 
tive domain. This conflicts with the overall educational goal of fostering 
student thinking. Judges felt that many exercises possessing content 
validity couM be reworked to raise the cognitive level at which taJrl^' 
populations could respond. 



Prom the investigator's review of curriculum guides and social studies 
programs it seems that the majority of the exercises contained content 
fairly representative of that covered in most schools in the country. With 
regard to the released social studies exercises, it did not appear that the 
content of the exercises put any group at a great disadvantage. Of course, 
the type of response to particular exercises would and did vaiy depending 
upon one's culture or racial group as well as prior experience. However, 
diversity in response with regard to these exercises did not penalize the 
individual. 



Also, the exercises seemed to cover topics that were for the most part 
relevant to the neec's and interests of most students. Several exercises 
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dealt with prejudice and stereotypes as ««U as with some of the preblems 
facing majorities. 

A point wrthy or some consideration is that persons having difficulty 
in reading .night be at a disadvantage on those objective exercises group 
a^ninlstered. One did need so»e competence in reading to interpret the 
questions. 



ooaia] Studies Kxeroines, l/nreloased 
Objective I. Have Curiosity About Human Affairs 

Only five exercises existed to measure this objective and all were 
individually administered. The exercise represented all age levels, ex- 
cept age 17. The judges considered all the exercises to be vaUd as well 
as ex»plary and appropriate for the ages indicated. Once again, the ex- 
ercises stress student functioning at the lower cognitive levels; never- 
theless, both the cognitive and affective domains are represented. Also, 
the nature of the objective suggests having just basic knowledge and a ' 
willingness to attend. 

Objective n. Use Analytic^cientific Procedures Kffeotively 

In the unreleased social studies exercises, aW an identical m™- 
ber of e»rcises existed for this objective as with the released. .twenty- 
four for unreleased. twenty^even for released. The co*i„atio„ fot»at 
the only one not represented. This was also true with the relonsed 
exercises. The items «re primarily group administered and geared to the 
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a«o «„up,. Percentagewise, the unre.ease. oxe^^see ™. .etter 

vath regard to content validltv Pa^^i 

"«/ vaxzaiLy. Panel members onlv iuda«.H 
. ° "^"-^y judged one exercise 

a^^ ha.„. 00.., ^^^^^^^^ 

mat a, ,^,,3. , ^^^^^^^^ ^^^^^^^^ 

objective, these exercises loaded primrily at th« i„ 
but th.~ P'^naniy at the lower cognitive levels, 

but there «as a ai«ere„ce i„ that ei.ht exercises wore coded at the ' 
analysis level, no discernable affective level „as noted. 

Objective in. Ar, S^^itive^o^^^^^^^^^ ^^^^ 

TM, »jor ohjectiv, had onl, five exercises., ..iti„« 
— folate «p«se„ted. .he .,o.t. of exercises. L. 
^r. ^^ ad^nistered and «ere geared to each a.e ,ro„p. .he review 
P^X considered all exercises', valid and ex«.pla. 
the .«,s. It is enccuragin, to note that t« of the exercises wer. at 
he »«„itive level of anal^is. The affective dimension was not die- 

^ero in« on the factor of sensitivity. 

Objective XV. XSjS'lM^r '"^^^ Concent 

SiMUr to the released social studies exercises, a rel^tivel^ greater 
2" - -cis. were prepared for this objective. Specifical^; 
exercises ws„ p„,„, _ 

ehoice and group adnlnistered. All am 

_ , levels were represented. However, 

there was a significant nuaber of e3ter^i,». * ■, 

Thl. . ««rcises. tvmlve, Judged as not valid. 

Thl. pattern duplicated most for the released s„m»i . 

ne released social studies exercises. 
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Also, the cognitive levels of these exercises ™re at the knowledge and 
conprehension levels of the cognitive taxonomy. Panel members concluded 
that no identification of the affective level possible «ith any degree 
of certainty. The panel members also felt that the item developers «ere 
again functioning under the assumption that questions designed specificai:y 
for nine-year^lds should be at the lower cognitive levels. 

Objective V. Have a^^^oned Oo-itment to the Values that Sustain a 

There were eleven exercises for this objective. Only the completion 
fo«at was not present. AU ages were covered in the exercise pool. Again, 
the judges classified aU exercises as possessing content vaUdity and 
appropriateness for teacher use. The cognitive emphasis was on both cob» 
prehem,ion and synthesis, and the affective level «as strongly represerted 
with eight exercises at the characterization level. In fact, the panel 
judges felt that some of these exercises exemplified the best atten^jts at 
assessing the affective domain. 

General Discussion of Unreleased Social Studies Exercises 

The unreleased exercises emphasize similar points, contents, e:,*- 
riences, concerns, attitudes, and interests as the released exercises. 
Much of what was said regarding the released exercises could be reiterated 
here. This is not surprising since bot:. released and unreleased exercises 
were drawn from the same exercise pool. 

Again, exercises that would be a«,„ered differently by various racial 
or ethnic groups are present, but it see«B that diversity in response was 
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accepted, fix>m the observation of the responses given by individuaa^ and 
considered appropriable. 

As p»vlous3y merrtioned. the exorcises overall require reading ekills 
and thus night put individuals with reading difficulty at some disadvantage. 

The overaU emphasis on the lower cognitive dimension is of some con- 
cert, but the judges did feel that most exorcises could be reworked to 
cater to high cognitive and affective levels. 



General Oomments Regarding Social Studies Exercises 
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Table 90 records the data related to both the released and unreleased 
social studies exercises. The table is a repeat of Table 40 and is pre. 
sented again to serve as the focus for a more detailed discussion. Perhaps 
the most Significant point to consider is that of the 1% exercises prenar«l 
for social studies. 85 per cent (164) of them Ke« considered valid by the 
investigator and the panel members. This should enable us to have some 
degree of faith in the exercises as being truthful In measuring what they 
state they are measuring. 

But just lookir« at the overall percentage of exercises can be mis- 
leading for not all objectives ar« represented by equal nuntoe.^ of exercises, 
and also the exercises emphasise different oomtiv. and affective levels. 
Of even more concern is the fact that several objectives are not repre- 
sented by any exercises or are represerted by exercises that have been 
Judged not valid. 

Objectives II and OT had significantly more exercises prepared} 51 
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and $3 exercises, respectively, ^-ftiether this suggests a hierarchy of 
importance regarding the objectives remains to be seen. Perhaps this 
.. loading is due to the objectives themselves | Objective II relates to wheth- 
er individuals use the analytic-scientific procedures. With the emphasis 
upon process today, perhaps this distribution of exercises reflects current 
curricular status. Hbwever, thisc inference is somewhat suspect, for if 
process were th^ emphasis, the exercises would not cluster at the lower 
cognitive levels, but would emphasize application, analysis, synthesis, and 
evaluation. Perhaps the number of exercises is related to the number of 
sub-divisions in the objectives. For example. Objective IV has six sub-di- 
visions. Also, this objective relates to the knowledge and understanding 
of the major social sciences i^ich entails a major dimension of social 
studies . 

But if NAEP has stated that social studies learning should be related 
to five major objectives, then all dimensions of these objectives should be 
represented by exercises. That no exercises exist for Objectives II B, 
II F, III C and V D and that only one exercise exists for Objectives III A 
and IV A makes it impossible to assess if individuals are demonstrating 
behaviors couched in the objectives; The absence of exercises or a minimal 
niimber of exercises makes i:-, insuperable to make any statement regarding 
whether individuals have certain understandings, commitments, or capabili- 
ties. Therefore, NAEP needs to either eliminate these sub-objectives or to 
create exercises for th«m. 

When one considers only the major categories of objectives, e.g. Ob- 
jective I, Objective II, and so foHh, these lack of exercises 
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aw not as evident, and persons ndght misinterpret assessment data to state 
that individuals at certain age levels possess particular behaviors when 
in fact no exercises existed to make such an assessment. For instance, no 
exercises exist for Objective 11 P "Use data and evaluative criteria to 
make decisions." However, when just considering the overall Objective II 
-Use Analytic^cientific procedures effectively," one might think that 
persons could use data to make decisions, yet there is no information 
gathered on which to base such a statement. 

This lack of exercises is a most serious defect in the assessment 
procedure and this investigator urges NAEP to develop exercises for each 
sub-objective or to eliminate *.hose sub-objectives lacking exercises. 
Also, this investigator would encourage that the overweighting and under- 
weighting of objectives be corrected or that a statement be issued that 
certain sub-objectives are more important than othera and should be wnk 
ordered* 

There are numerous ways to analyze the exercises, such as clustering, 
directionality, and difficulty level. These were not the concerns of the 
Task Two work force; however, it did appear that many of the exercises did 
represent clustering; but for the most part the clustering existed within 
items, that is, the sub-components of exercises related to a particular 
stimulus* 

Regarding directionaUty, NkEP, in discussing the exerL^ises, stated 
that all exercises are scorable or must be scorable in a desired direction, 
which means that they must possess directionality. All exercises exhibited 
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thi., although in many instances great dive„,ity of responses «as 
accepted. In those exercises stressingth. affective domain, directionality 
^ ™.re difficult to deterMne because of the open-endedness of the exer- 
oxses. As for directionality that n^asured change over time, the panel 
could not dete^ne this f,o„ considering exercises adMnistered once. 
But if exercises ax, to be given at various tin,e pertods then directional- 
ity can be assessed to so«e degree. l„ national assess^nt. one is inter- 
»ted not only in the current level of social studies understandings, but 
also in Whether over ti«e the level of achievement, skill use. and atti- 
tudes ifflprove. 

Age appropriateness is related to the difficulty factor of the exer- 
cises. In the opinion of the Judges, the mjority of exercises, both 
released ami unreleased. were of proper difficulty level, but in some ' 
instances, exercises «ere too difficult for nine-yearK,lds. I„ some in- 
stances ,*ere a question of age appropriateness appeared regar^„g the 
«pper age lev»l«. the panel members felt that the exercises «sre too 
easy or could be used at a lower level than indicated. 

Citizenship Exercises. Released 

The next two sections deal with citizenship exorcises, both released 
and unreleased. Summa^ data for these exercises is presented in Table 91 
i*ich is found on page. Ill and 112 . This table repeats the information 
presented in Table 89. The reader may wish to review these tables before 
reading the following paragraphs. 
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Objectivo I. Show Concern for the Welfare and Dignity of Others 

Ten exerciees represented this objective «ith all foi»ate used and 
the mjorlty of the exercise, administered individually. The iteiK, 
clustered at the three upper age levels. With regard to content val- 
idity, the panel me<»bers considered only five exercises to be vaUd. 
This see™ unusually lo„, and perhaps indicates the difficulty of devel- 
oping items that measure «hether individuals show concern for others. One 
would e:^ect that in addition to the cognitive domain, that exerelses de- 
signed for this Objective would cater to the affective domain. Ho«ver, 
only four exercises exemplified a level of the affective do^n and all 
exercises centered on the knowledge level. 

Objective n. Si9.port Rights and Freedoms of m Individuals 

In all, had six exercises to assess this objective. The majo.^ 
ity Of them used the co*ination fomat and were administered individualjy. 
Exercises catered to all age levels, «lth a loading on age 13. Taml 
meaber, dete«l«ed that five e»reises were valid and exemplaty. Both 
domain, wore represented, with the cognitive domln st.^ssing the knowi- 
edge and comprehension levels, while the affective domain emphasized the 
three upper affective levels. 

Some of the exercises could have had their cognitive and affective 
level, raised by presenting situations in which individuals would have to 
demonstrate that they had indeed supported rights and freedoms. This concern 
«as evidenced when the review member considered that only th«e exercises 
had contact vaUdity. that Is. they represented situations in which the 
context »s appropriate for having individuals truly demonstrating the 
behavior. 
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Objective III, Help Maintain Law and Order 

The four exercises developed for this objective divided between the 
toting and coiiijination foxmats. All were individually adrd-nistered and 
represented all age levels. The panel classified all exercises as vaUd 
with regard to the objective being measured and considered aU to have 
exemplary qualities. 

It seems that with the current concern for law and order that more 
exercises might have been developed and that they could have been at 
levels other than the cognitive level of comprehension. A raising of the 
cognitive level wuld require from individuals more than just an understanding 
of the need for law and order and would actually assess whether persons in- 
volved themselves with maintenance of law and order. One exercise did 
qualify as being at the organization level of the affective domain. 

Objective IV. Know the Main Structure and Functions of Our CJovemment 

This objective had a heavy loading of exercises, eighteen in all, 
with the majority utilizing the completion format. Itost exercises were 
administered individually and centered on the mid age groupings of 13 and 
17. In this instance, we again discuss another large grouping of exer- 
cises deemed to a^ck content validity.' The judges thought that the exer- 
cises were geared only to whether individuals possess knowledge of our 
government and could comprehend the basic components. In assessing this 
citizenship objective, one should be concerned with whether individuals 
are cognizant of and can recognize basic assumptions of our government 
and can utilize such understanding in specific situations. 
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Panel „e*or, considered only eight of the eighteen e^rcisea ae good 
.:«,ple3 .or teachers, .e. Also. „.„hers noted that n^e^us exercises 
were not appr^p^ate for the ages indicated. So™e of the exercises «ere 
too oas. for 17-year-olds and adults. This faiUng ^ght be due to the 
fact that the questions «ere at the lower cognitive levels. Seventeen- 
year^lds and adults can certainly fu„etion at higher levels. A trend 
noted With regard to the social studies exercises also was cedent in 
these Citizenship exercises for this objective, that is. a tendency to 
sear the exercises for the ages of , and 13 to the lower cognitive levels, 
dually the Wlodge level. This see. to indicate that exercls. devel- 
oped, perceive teachers i„ the schools as Just stressing facts. Of cou„,. 
this »ay reflect the present situation »ore than ^„y ^ .^e willing to 
.a«t. The emphasis „.y also reflect the popular belief that if citizens 
know a few facts, that is all that is necessa.7. 

•I 

Objective V. 3e* ^^ty ^„ve„e„t Through Active, .e»oc«.tic 

aeven exercises assessed this objective. ,<ost were administered 
individually and used the co.*i„atio„ fo^at. Although all age levels 
were involved, the ^Jority of the exercises were for adults, only one was 
for nine-year-olds and t« for 13-year-olds. This reflects somewhat the 
nature of the objective, but it does see™ to suggest that co„y ^ 
defined as being outside the school and therefore not approprt.,e for 
young Children. Further, it suggests that there is r^rhaps a lack of ac- 
tivities in the schools in which children can function in de»,cratic ways. 
If this is so, it suggests a need to rethi* our goals, for certainly, young 
persons require experiences in co«unity involvement both inside and o,*side 
the school. 
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The investigator and the panel „eM,e. olassifiea .est e^rcisos as 
v^Ud and exe;npla:7. It bo of so^e interest that several levels of 
the cognitive do..i„ ^re represented and that see exercises loaded on 
the highest affective level. This is consistent with the nature of the ob- 
octive. for if one is to he involved, one has to de.o„t..e or indicat, 
bohav^ors that reveal a type of conduct characte..ed h. certain value sets. 
Objective VI. unde^ta^ p„,i,^ Inte^ational Relation. 

Wise, designed to assess this objective nu^ered five, with an 
exercises being essa. fo»at and individuals^ adMni^tered. The exercise, 
concentrated on the cognitive levels of .„o„ledge and comprehension, as 
w^I as s^hesis. The affe<.ive level of cha...ert.atio„ „as represe^^d 
aUo The t.ndon,^ to design questions for ni„e-^..ids at the .novledge 
level was apparent, 

elective vn. .tio^^^^^^^ ^^^^^^^ 

«» had ten exercises for this objective vdth all fo«ts used except 
the completion. The „a„„er of administration «as divided aW e-^li, 
between individ^l and g..p. and all age levels we. .p^eented. .udge. 
considered six exercises .lid and four not valid. It is inte^sting to 

note that the cofl?)rehension level had the n.r,«f . 

xovex naa the most exercises--six. Thus it 

that «^ Of the exercises assure whether individuals co«„ded 
^t rational action and thought a., hov^ver, three exercises «re coded 
- having the valuing and organisation levels of the affective do^in 

fudges felt that at least one exercise in this grouping, exercise la. 
a racial bias. Ce^ainl, the entire .aim 3ocial problems «ul4 
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be viewed differently by ?!rinoritie8 than by the majority. 

Objective VIH. Take Responsibility for Own Personal Development and 

ObHgations 

This objective had four exercises. The exercises inwlved the formats 
of multiple choice and combination. Four were administered individually 
and all age groups were represented. Of the four exercises, panel members 
judged only two as valid and exemplary. Part of the tlifficulty may be 
that three of the exercises were at the knowledge level. If one wishes to 
determine if individuals are assuming responsibility, one needs to have the 
majority of exercises above this level and specifically relating to the 
upper levels of the affective domain. Only two of the exercises were con- 
sidered to represent affective categories, valuing and organization, re- 
spectively. 

With regard to exercises at this level, one may find that individuals 
from ndnorities and/or disadvantaged groups may not be able to respond 
positively. Also, some individuals are so concerned with present-day 
realities, that they do not have time for planning for the future. The 
investigator and panel members concluded that this objective, especially 
the sub-objectives, reflect middle class value standards. The judges did 
not classify these values as good or bad, but the reader should note that 
these values are being emphasized in this particular objective. 

Objective 32. Help and Respect Their Own Families (Ages 9* 13* 17) 
Nurture the Development of Their Children as Future 
Citizens (Adult) 

The six exercises prepared for this objective were equally divided 
among the completion and ooirbination fomats and among individual and group 
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adm^inistration. All age levels were treated. However, only three exer- 
cises qualified as valid and exemplary. Many of the exercises just asked 
for descriptive types of information rather than whether individuals 
helped or respected their families. For example, one exercise queried if 
children had brothers or sisters. Certainly, one cannot assess if one 
helps or indeed respects their families from such a question^ 

It is significant also that only six exercises were used for this ob- 
jective. This objective can bring home whether individuals are putting 
into their actions much of what citizenship is all about j being the 
effective citizen and having concern for individuals* 

General Discussion of Released Citizenship Exerciser 

Perhaps the most significant point with regard to the citizenship ex«- 

i 

ercises released is that over 30 per cent bf the exercises did not possess 
content validity. This is a different stoxy from the social studies ex«- 
ercises* Also, the fact that the exercises stressed the lower cognitive 
levels and did not emphasize the affective domain seems inconsistent with 
the overall thrust of citizenship education. VH.th regard to the cognitive 
domain, the dominant use of exercises stressing the lower cognitive levels 
for populations at age nine is not desirable. As with the social studies 
exercises, certain objectives seemed to be favored with regard to number of 
exercises. For example. Objectives I "Show concern for welfare and dignity 
of others," IV "Know the main structure and functions of our government, ♦» 
V "Seek community improvement through active, democratic participation," 
and VII "Support rationality in com^mnication, thought and action on 
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social problems" had the most exercises J 10, IS, 11, 10, respectively. 
Again, one queries why this is so. Does this imply the Objective IV, "Know 
the main structure and functions of our government" is considered paramount 
to being the knowledgable and effective citizen? It should be borne in 
mind that this objective also had a large percentage, almost fifty per cent, 
of the exercises in the not valid categoiy. 

Overall, the content stressed in the exercises appears to parallel 
that which would be covered in most schools in the coxintry. However, 
several exercises may be biased in favor of the middle-class value struc- 
ture. This bias may put individuals from minority groups at a disadvan- 
tage. Of course, if the purpose is just to assess, then one could argue 
that an individiial is not disadvantaged by being unable to respond in an 
appropriate manner. But, appropriate manner is the point that generates 
arguments. Certainly, individuals from various minority groups will per- 
ceive the exercises differently from individuals growing up in middle 
class white America. However, one might argue that the regional differ- 
ences might create in individuals as great a difference in perception. 
Rather than fault the exercises on this point, one might just caution the 
reader and educators interpreting the results to keep such information in 
ndnd. 

In general, there is legitimate reason for concern with the large 
number of non-valid exercises and also with the dominant enq)hasi8 on knowl- 
edge and con^rehension. Citizenship is generally regarded as something 
more than remembering facts and comprehending what is read or said. Cit- 
izenship is using one's information and recognizing how one can apply this 
information to function effectively in this country and world. 



TASK II 

As a group, the investigator and the panel members are not as impressed 
with the released citizenship exercises as with the released social studies 
exercises* 

Citizenship Exercises, Unreleased 

Objective !♦ Show Concern for the Welfare and Dignity of Others 

Objective I had twelve exercises representing all four formats. Ten 
exercises were individually administered and all age levels were represented, 
with emphasis at ages 13 and adult . Panel review members judged seven ex- 
ercises as having content validity. Only six exercises qualified as being 
exenqjlary. Vttth regard to the cognitive domain, the exercises centered on 
both the knowledge and comprehension levels. The major investigator felt 
that many items could have been rewritten to make them valid for the ob- 
jective intended and also to raise their cognitive level. Also, since the 
objective was related to individuals showing concern for others, one would 
have anticipated more of the exercises having clearly discernible affec- 
tive levels. The major reason for considering some exercises not exemplary 
was that they did not measure the intended objective; however, some items 
were valid, but could have been more stimulating or been at a higher in- 
terest level for individuals. 

Objective II. Support Rights and Freedoms of All Individuals 

Five exercises were present for this level. I'fultiple choice was the 
dcadnant format. The majority of the exercises were group administere'*'., 
and they stressed all of the age levels. The review members ranked only 
three exercises as valid. All of the exercises grouped around the two lowest 
cognitive levels. 

ERIC 
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It is of some interest that one of the exercises in this grouping, 
item # 16, unreleased, was considered by the major investigator and all 
members of the review panel to be one of the best exercises of all those 
developed. This exercise had content validity with regard to this objec~ 
tive and two others, V and VII, Furthermore, it represented both the cog- 
nitive and affective domains, analysis and organization, respectively. The 
review panel classified the exercise as most exemplary in that it utilized 
a meaningful format, combination, and allowed the respondent to become in- 
volved in reacting rather than just functioning from memoiy. 

Objective III, Help Maintain Law and Order 

This objective had four exercises, three of which were essay formats 
and one multiple choice. All were individually administered with repre- 
sentation for all ages, panel members and the major investigator dis- 
cerned all exercises to be valid and exemplary. However, the members 
thought that the exercises unduly stressed the lower cognitive levels, 
although one exercise was considered to represent the organization level 
in the affective domain. 

Objective IV. Know the Main Structure and Functions of Our Government 

This objective had a great many exercises prepared for it— twenty-eight. 
All formats were represented and exercises were about equally distributed 
between individually or group administered. All age levels were stressed, 
with almost equal numbers for ages 13, 1? and adult. As with the released 
items for this objective, many exercises were not valid. In fact^ fifty 
per cent (14) of the exercises failed to qualify as having content validity. 
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Judges decided that even fevrer exercises warranted being classified as ex- 
emplary or appropriate for the targeted ages. A main criticism of the 
judges, vfith the concurrence of the investigator, was that the exercises 
measured only knox-fledge of specific facts and such a cognitive level was 
not consistent with whether individuals really comprehended their govern- 
ment, its function and purposes. This reaction relates to the fact that 
citizenship functioning is something more than the mere regurgitation of 
facts. 

Also, there was a repeat of the tendency for exercises designed for 
the nine-year-olds to stress knowledge. Overall, one could say that all 
of the items tended to emphasize the lower cognitive levels. Many of the 
exercises were considered too easy, especially for the 17-year-olds and 
adults. Some of the exercises were poor examples because of the use of 
words that might not be clearly understood by the targeted populations. 
Some exercises, e.g., exercise 6, in this grouping violated rules of item 
construction such as making the correct discriminator unusually long in 
relation to the other options. The major criticism of this grouping of 
exercises is that the exercises overwhelmingly test for knowledge of spe- 
cific facts. 

Objective V. Seek Community Improvement Through Active, Democratic 
Participation 

Six exercises were prepared for this objective. All formats except 
completion were used and the exercises were administered equally between 
individual and group. The two upper age levels were the only ones repre- 
sented. The review panel found the majority, five, of the exercises to be 
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valid and appropriate for the intended ages. Both taxonomy domains were 
represented. In the cognitive domain there was a spread of representa- 
tions, while in the affective domain, the exercises concentrated at the 
upper level, characterization. 

This grouping had one of the outstanding items as well. It was Item 
IS, which stressed the cognitive level of analysis and the affective level 
of characterization. The prime reason for the very favorable ciassifica- 
tior. was that the exercise appeared likely to involve the respondents in 
reacting to the question. 

Objective VI. Understand Problems of International Relations 

Of the four exercises created for this objective, panel members 
judged all to be valid. All age levels were represented with most of the 
exercises being administered individually. All formats were used except 
the completion one. As is quite typical of these citizenship exercises, 
there was a heavy loading on the lower cognitive levels. 

Objective VII. Support Rationality in Communication, Thought and Action 

on Social Problems 

In total, ten exercises were developed for this major objective. All 
fonriats, excepting the completion format, v/ere used with the exercises, 
most of which were administered individually. There was almost equal dis- 
tribution of the exercises among the four age levels. Again, panel members 
noted only six exercises as having content validity. The cognitive levels 
of these exercises centered at the comprehension level which seemed to offer 
additional evidence that the majority of the exercises favored just 
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comprehension of citizenship factors rather than possessing the ability to 
use and act upon information relating to citizenship. It seems that an ex- 
ercise designed to assess whether one supports the process of rationality 
should have the upper levels of the affective domain represented. However, 
judges dj.d not code any exercises as having a clearly identifiable affec- 
tive component. 

Objective VIII. Take Responsibia.ity for Ovm Personal Development and 

Obligations 

The review panel classified five exercises relating to this objective. 
All used the combination format and four were administered on an individual 
basis. All age levels were represented. Even though there were few ex- 
ercises, members rated all as valid and exemplary. It became evident to 
the judges that in most cases t*here many of the exercises were valid in 
citizenship, there usually was also a higher incidence of affective levels. 

Objective IX. Help and Respect Their Own Families (Ages 9, 13, 17) 
Nurture the Development of Their Children as Future 
Citizens (Adults) 

The review of the exercises relating to this objective revealed the 
only instance in which panel members judged all of the exercises developed, 
only four, invalid. Panel members considered the questions to be more on 
the nature of an inventory rather than assessing citizenship behavior. The 
weakness of the exercises in the unreleased category parallel a similar 
weakness discovered in the released citizenship exercises. Thus, judges 
felt that this objective required a totally new set of exercises. 

General Discussion of Unreleased Gitisenship Bxercises 
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As was true with the released exercises, the panel members judged ap- 
proximately thirty per cent of the exercises to be not valid. It is some- 
what surprising that the percentages would be so similar for both the re- 
leased and unreleased exercises. 

Part of the difficulty may be that the exercises centered on measuring 
knowledge and comprehension and neglected the upper levels of the cc^rnitive 
domain and many of the levels of the affective domain. Again, it appears 
that the item developers consider that citizenship to a large extent is 
equated with just knowledge of facts relating to presidents, governmental 
agencies, dates, issues, and concerns. However, this investigator would 
maintain that even if targeted populations answered all questions satis- 
factorily, they would not have demonstrated to a great degree liiat. is 
meant by effective citizenship in today's education literature. 

Overall, the content of the exercises seemed relevant to the area of 
citizenship education. Lest this seem inconsistent with the above com- 
ments, the investigator feels that many of the exercises focused on rele- 
vant content areas, but failed to assess if individuals coiild process the 
content at the upper cognitive levels or could demonstrate whether they 
valued some dimensions of citizenship behavior and had incorporated these 
values into their overall behavior. 

The exercises reflect some norms for the ftinctioning citizen in the 
United States. Individuals from minority p;roups may respjond differentially 
but, it depends upon how educators interpret the data as to whether the 
persons from minority groups are at a disadvantage. Basing one's judgment 
more on the cognitive-affective levels rather than content, it appears that 
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the exercises, as a group, stress a "past" interpretation of the good 
citiaen, one who can cite names and dates, and list various legislation. 
It seems that even the exercises having content validity were valid only 
with regard to measuring the lower levels of cognition suggested in the ob- 
jectives, l-lany of the exercises should be rewritten to assess if individ- 
uals at the various age groups fully discern the basic reasons for the 
political scene, fully realize why it is necessary to show concern for 
one»s fellow man, and completely comprehend generalizations relating to 
the citizenship dimensions of man. Certainly, schools need information 
on these matters if educators are to possess data useful for adjusting and 
creating educational prograans dealing with citizenship, 

/ 

General Comments Regarding Citizenship Exercises ^ 

Observation of Table 91 provides all data for both released and un- 
released citizenship exercises. As is true with the social studies exer- 
cises, the citizenship exercises, both released and unreleased, favor cer- 
tain objectives. Objective I "Show concern for the welfare and dignity of 
others" had a total of 22 exercises prepared while Objective 17 had 46 ex- 
ercises. The emphasis on Objective IV, which deals with the main structure 
and function of our government, suggests that citizenship, as conceived by 
the exercise developers, is primarily political in emphasis. Citizenship 
may be equated with political socialization. However, the overall objec- 
tives would suggest a broader interpretation of citizenship that extends 
beyond competence in matters governmental. Certainly, educators need to 
have the total dimension of citizenship assessed by these exercises. It 
would be important to know if the number of exercises per objective 
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represents a conscious plan or is the result of just how the exercises 
happened to fal3. as developed and approved. 

This overweighting of some objectives and underweighting of others is 
a problem that needs attention. Alao, several citizenship objectives had 
no exercises at all: Sub-objectives IB, IE, III C, III D, IV P, V A, 
V F, VI A, and VII E, Also, there were many objectives represented by- 
only one exercise; Sub-objectives I F, I G, III E, III F, IV A, V D, VII C. 
In one instance the sole exercise was judged as not having content validity. 
Sub-objective I G. 

NAEP has by its objectives stated that citizenship basically concerns 
certain things and has mentioned that the goal of national assessment is 
to measure if individuals have attained those certain knowledges, attitudes, 
skills* However, it has a large percent age of sub-objectives not repre- 
sented by exercises and therefore lacl<s the basis to make statements that 
individuals have or have not attained certain understandings, attitudes, or 
skills* As mentioned with regard to the social studies exercises, this 
investigator urges NAEP to either eliminate the sub-objectives for which no 
exercises exist or to create valid exercises* Also, the investigator urges 
that NAEF attend to the problem of a more equal distribution of exercises 
among several objectives* 

Not only did the investigator and the panel members find these gaps in 
exercise existence, but of the total pool of exercises developed, 152, only 
93 were judged as having content validity— or 6l per cent* One needs to ex- 
ercise extreme caution in assessing the level of citizenship functioning of 
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individuals when 39 per cent of the exercises are not valid. Even fewer ex- 
ercises were considered exemplary— only 55 per cent. 

vath regard to clustering, many of the exercises utilized the combina- 
tion format in which a series of sub-questions were based on one set of 
stimulus material. 

All exercises that measured the cognitive dimensions had direction- 
ality in that they had correct or preferred answers. However, r.he issue of 
directionality is more difficult in those exercises dealing with the af- 
fective domain in that such exercises do not have a correct or desired 
anst<er in all instances. Of course, in some of the exercises a preferred 
direction of response could be inferred. 

As for difficulty level, the investigator and his panel members con- 
sidered most vaHd exercises at appropriate difficulty levels for target ^ 
age groups. In situations where the panel indicated an exercise as not 
appropriate for the age it was usually because the exercise was too easy 
for the group rather than too difficult. 

As a whole the investigator and the review panel regarded the citizen- 
ship exercises as not measuring up as well as the social studies exercises. 
As previously stated, thirty-nine per cent failed to qualify as valid. 
Additionally, many of the exercises having content validity vrere only as- 
sessing the lower cognitive dimensions. Many of the exercises reflected 
an interpretation that schooling is synonymous with fact gathering, re- 
gurgitation of facts, and paraphrasing information. 



' TASK II 



BEST COPY mma 



Final Commenta 

The investigator and the panel members spent numerous hours carefully 
revievdng exercises and making judgments as to v/hether exercises were rdp<" 
resentative of the content or the universe of content of the property 
being measured. As a vihole, the panel concurred with the investigator 
that the social studies exercises as a total group, both released and un- 
released, were superior to the citizenship exercises, both released and 
unreleased, in that a greater percentage (S5 per cent) of social studies 
exercises possessed content validity. 

But, even if one were to have found all the exercises possessing con- 
tent validity, one still needs to query himself regarding the concept of 
national assessment. If one is opposed to the concept, then no amount of 
valid exercises will win him over. However, many believe that national 
assessment has much to offer. Certainly educators require some feedback 
in order to maintain what is effective in the schools both in the realms 
of curriculum and instruction and to eliminate and/or modify that which 
does not seem to measure up to our objectives* 

But the task is not that simple. Assuming that all of the exercises 
are valid and measure the upper levels of both the cognitive and affective 
domains, one still cannot determine precisely if schools are to be praised 
or blamed for the current levels of understanding and functioning of in- 
dividuals in social studies and citizenship. Certainly, the schools are 
not the only institutions in society that educate. Individuals do not 
obtain all of their understanding of their world or their attitudes towards 
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others just in the schools. This certainly is tn^e with seventeen-year- 
olds and adults. Therefore, national assessment can tell us what are the 
levels of xmderstandings, skills, attitudes, etc. of various segments of 
our population, but it cannot tell us that these levels are entirely the 
results of good or poor schoolins. Perhaps educators can agree to the 
need for assessment, but our challenge is what we do with the results if 
we accept them. Of course, from the investigation of Task II, some adjust- 
ments will need to be made in many of the exercises to afford educators 
with information beneficial as input to their decision-making regarding 
what schools need to do to enable individuals to become truly effective 
persons, citizens of this and the next century. 
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APPENDIX A 

Specific Items Gonsidered Not Valid 
an d/or Not Exemplary 



fixercise Area; SSR 



Not VaUd 

10 A (101020) 
1 A (101022) 

3 A (Obj. n A) 

22 A (203002) 

15 (0b;J. II D) 
6 A (ObJ. 11 D) 

4 (A0200d) 
15 A, D (403001) 

4 (403011) 
13 (Obj. II E) 
13 (404006) 

23 (4OAOI2) 

15 (4O5OO6) 

Id (405011) 

20 (40504a) 

16 (4O6OO6) 

11 (4O6OO9) 
4 (427001) 



Not Exemplary 

1 A (101022) 

3 A 

22 A (203002) 
6 A 

4 (4020O8) 
15 A, D (403001) 

4 (403011) 

13 (404006) 

23 (404012) 
15 (405OO6) 

20 (405040) 

Appropriate for 
knowledge objective 

u (406009) 

4 (427001) 
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Speeifie Items Considered Nob Valid 




and/or Not Exemplary 






Exercise Areas SSU 




• 


Not Valid 


Not SxeniDlary 




6 (i!i03005) 






10 (403026) 


10 (403026) 




2 (403002) 


Good for different 
obiective 




24 (404013) 


24 (404013) 




16 (404015) 


16 (404015) 




10 A, B 


10 A, B 




2 (405010) 


2 (405010) 




7 A, B (4050L2) 


Good for another 
objective 




3 (405007) 


3 (405007) 




10 (405021) 


10 (405021) 

21 




5 (4O6OO7) 


5 (406007) 
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III 


Specific Items Considered Not Valid 


and/op Not Exemplary 




Sxercise Area: GR 




Not Valid 


Not Exemplary 


17 A, B, C, D, E, P (Obj. 1 A) 


17 A, B, C, E, P 


13 A - P (Obj. I A) 


13 A - P 


1 A, B, C (Obj. I D) 


1 A, B, 0 


11 A, B, C (Obj. I D) 


n A, B, C 


3 A - D (Obj. I D) 


3 A - D 


11 (Obj. II B) 




19 (Obj. IV B) 


19 


6 (Obj. IV B) 


6 


Id (Obj. IV B) 


18 


2 A - E (Obj. IV B) 


2 A - E 


17 (Obj. IV B) 


17 


11 (Obj. IV B) 


11 


3 A - b (Obj. IV C) 


3 A - D 


19 (Obj. IV D) 
1 (Obj. IV E) 


Appropriate for 
another objective 


5 A, B, C (Obj. IV D) 


5 A, B, C 


E 12 (Obj. V C) 


B 12 


E 11 (Obj. V C) 


Appropriate for 
another objective 


12 (Obj. VI) 


12 


8 A, B (Obj. VII A) 
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Specific Items Considered Not Valid 
and/or Not Exemplary 



Exercise Area: CR - Continued 



Not Valid 

4 (Obj. VII A) 

10 (Obj. VII A) 
14 (Obj. VII A) 

7 (Obj. VII A) 
16 (Obj. VII D) 
16 (Obj. VII F) 

11 A, B (Obj. Vm A) 

3 A - C (For Adults) (Obj. VIII A) 
3 A - B (For Adults) (Obj. VIII A) 
3 A - C (For 9-year olds) (Obj. IX A) 
aO A - E (Obj. IX B) 

5 A - C (Obj. DC B) 
14 (Obj. IX B) 



Not Exemplary 



14 
7 



3 A - C 



3 A - C 



5 A - C 
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Specific Items Considered Not Valid 
and/or Not Exemplary 



Exercise Area: CU 

Not Valid 



Ncc Exemplary 



22 (Obj, I A) 


22 




3 A - P (Obj. I C) 


3 A- 


P 


11 A - D (Obj. I D) 


11 A - 


D 


9 A - D (age 13) (Obj. I D) 


9 A - 


D 


9 A - D (age 17-Adult) (Obj, I G) 






1 (Obj» IV 3, inul» choice format) 


1 




1 (Obj, IV B, completion format) 


1 




2 (Obj, IV B, raul, choice format) 


Z 




2 (Obj, IV B, con?)letion format) 


2 




10 (Obj, IV B) 


10 




15 (Obj, IV B, Ages 13, 17, Ad.) 


15 




15 (Obj. IV, age 9) 


15 










9 (Obj. IV D) 






17 (Obj. IV C) 


17 




d (Obj, V B) 


d 




Id (Obj, VII A) 


18 




9 A, B, C (Obj. VII A) 






7 A - D (Obj, DC A) 






e A, B (Obj. IX A) 


d A, 


B 


14 A - P (Obj. K B) 


14 A • 


. P 


16 A « 0 (Obj. IX B) 
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APPENDIX B 

Biographical Sketch 

Francis P. H\uikins, Ph.D. 

Dr. Hunkins received his B.S, in Education in I960 from Salem State 
College, Salem, Massachusetts; his M.Bd. in 1963 from Boston University, 
Boston, Massachusetts J and his Ph.D. in 1966 from Kent State University, 
Kent, Ohio. Since 1966, he has been at the University oi' Washington where 
he is Professor of fiducation. 

Dr. Hunkins* main areas of professional concentration are general cur- 
riculum, social studies education, and instructional strategies, with 
special emphasis upon questions and questioning strategies. It is within 
the area of questions that Professor Hunkins* research interests primarily 
lie, and he has conducted research funded by the U.S. Office of Education 
dealing with the effects of various types of questions on the thinking and 
achievement of students in element axy social studies. Dr. Hunkins also is 
active in giving numerous woricshops throughout the country to teachers and 
prospective teachers dealing with questions and questioning strategies. 

He is the atzthor of the fifth grade textbook and toacher*s guide, 
ASKING ABOUT THE U.S.A. AND ITS NEIGHBORS (American Book Company, 1971); 
a college textbook QUESTIONING STRATEGIES AND TECHNIQUES (Allyn and Bacon, 
1972), and eo-author with Patricia F. Spears of the ASCD monograph, SOCIAL 
STUDIES FOR THE EVOLVING INDIVIDUAL (ASCD, 1973) » and an author of a chap- 
ter dealing with questions In the upcoming 1974 NCSS Yearbook. In addi- 
tion to this writing, Dr* Hunkins has authored over thirty-five articles 



TASK II 



VII 



in the areas of social studies education, general curriculum, and teaching 
strategies. 

Professor Hunkins is active in The Association for Supervision and 
Curriculum Development, the National Council for the Social Studies, and 
the American Educational Research Association, He served on the Advisory 
Board of Social Education from 1971 to 1973. He has presented numerous 
papers on his research on questions and other topics to these organiza- 
tions, most recently conducting an action lab at the ASCD national conven- 
tion in 1973. 

In addition to giving \«rkshops on questions. Dr. Hunkins works with 
school districts around the country on matters relating to curriculum 
development and implementation. 
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Biographical Sketch 
Thresa B. Gieger 

Ms. Thresa Gieger received her B.A. from Arizona State University, 
Tempe, Arizona, in 1960j her M.A, from Arizona State University in 1964; 
and is currently wAing on her Ph.D. at the University of Washington, 
Seattle, Washington. In addition to her F,raduate vioi^ she is an instruc- 
tional specialist in the Portland, Oregon Public Schools* 

Ms, Gieger has been active in providing consultant services to school 
systems outside her own, and she has taught graduate classes in general 
curriculum and instruction and has given numerous speeches and presenta- 
tions to groups of teachers. In the summer of 1972, she received a U.S. 
Office of Education grant for a three-month resident study of the Yugoslav 
Educational System. 

Ms* Gieger i3 active in NCSS and has served on committees and has 
been a participant in state and national conventions. She also has con- 
tributed to a volume on Yugoslav education to be pubHshed in the near 
future. 
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Biographical Sketch 
Jack S. Thompson 

Mr, Jack S. Thompson received his B,A, in 1959 from Western Washing- 
ton State College, Bellingham, Washingtonj and in 1962 received his M.A. 
from Stanford University, Stanford, California, vdth a major in Social 
Studies Bkiucation, Currently, he is the Social Studies Coordinator for 
the lienton School District, Renton, Washington. 

As part of his responsibilities as social studies coordinator, he has 
assisted in the development of numeroim social studies materials and has 
been involved in program evaluation. He has conducted various inservice 
programs dealing vdth improving teachers* instruction as viell as being 
involved in ititem and student teacher programs. 

Professionally, Mr. Thompson is a member of the Washington State 
Council of Social Studies Coordinators and is the president for the 1973-74 
tenn. He also is a member of the Pu^^et Sound Council for Social Studies 
and a member of the National Education Association. 
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Biographical Sketch 
Arthur £. Ware 

Mr, Arthur Ware received his B.A. in I963 from Whitworth CoHege, 
Spokane, Washington; his M.A. from Trenton State College, in Trenton, 
New Jeraeyj his M.Div. from Princeton, Princeton, New Jersey in 1963; 
and is currently a Ph.D. Candidate in the Department of Curriculum and 
Instiniction at the University of Washington, Seattle, Washington. Along 
with his studies, Mr. Ware is the social studies coordinator in the Bellevue 
Public Schools, Bellevue, Washington. 

His professional concentrations in social studies education and cur- 
riculum have involved him in coordinating the development of a K*-9 social 
studies program for his school district. He also has trained administrators 
in Bellevue in Flanders Interaction Analysis. 

Additionally, he is active in the N&tional Council of Social Studies 
and has made presentations at the 1971 and 1972 national conventions. He 
also has set*ved as a consultant in social studies for teachers and admin- 
istrators in various school districts in the Nbrthwest as well as British 
Columbia and Alberta. In June, 1973 he assisted in training administrators 
in the new social studies programs for the National Science Foundation at 
Stanford* 



